Sep 07 08:57:02.893570 ip-10-0-138-157 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 07 08:57:02.893584 ip-10-0-138-157 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 07 08:57:02.893595 ip-10-0-138-157 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 07 08:57:02.893906 ip-10-0-138-157 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 07 08:57:13.142521 ip-10-0-138-157 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 07 08:57:13.142539 ip-10-0-138-157 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c9c6250c36c44ec0a2341311484a23fc --
Sep 07 08:59:01.921571 ip-10-0-138-157 systemd[1]: Starting Kubernetes Kubelet...
Sep 07 08:59:02.282981 ip-10-0-138-157 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 08:59:02.282981 ip-10-0-138-157 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 07 08:59:02.282981 ip-10-0-138-157 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 08:59:02.282981 ip-10-0-138-157 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 07 08:59:02.282981 ip-10-0-138-157 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 08:59:02.283578 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.283512    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 07 08:59:02.289253 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289238    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:59:02.289253 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289252    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289256    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289259    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289263    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289267    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289270    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289273    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289276    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289291    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289303    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289306    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289309    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289311    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289314    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289317    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289320    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289323    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289325    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289328    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289331    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:59:02.289332 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289334    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289337    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289339    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289342    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289345    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289348    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289350    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289353    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289355    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289358    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289361    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289363    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289366    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289368    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289371    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289373    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289376    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289379    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289381    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289383    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:59:02.289876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289387    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289389    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289392    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289394    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289397    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289399    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289402    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289405    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289407    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289409    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289413    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289415    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289418    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289420    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289423    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289426    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289429    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289431    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289434    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:59:02.290402 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289437    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289442    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289445    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289448    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289451    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289454    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289457    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289460    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289463    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289466    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289468    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289476    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289478    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289481    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289485    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289489    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289492    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289495    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289498    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:59:02.290876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289500    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289503    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289506    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289508    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289511    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289513    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289516    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289519    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289523    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289527    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289530    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289534    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289536    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289539    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289541    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289544    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289547    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289550    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289552    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:59:02.291361 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.289555    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290223    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290230    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290232    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290235    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290238    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290241    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290243    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290246    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290249    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290252    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290254    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290257    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290260    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290262    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290265    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290270    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290273    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290276    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:59:02.291820 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290293    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290296    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290300    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290303    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290306    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290309    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290311    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290314    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290317    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290319    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290322    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290324    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290327    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290329    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290332    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290335    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290337    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290339    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290342    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:59:02.292302 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290344    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290348    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290351    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290354    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290357    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290360    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290362    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290365    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290367    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290371    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290374    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290377    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290380    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290382    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290384    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290389    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290391    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290394    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290398    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290400    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:59:02.292767 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290403    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290408    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290411    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290415    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290418    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290420    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290423    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290426    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290429    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290432    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290434    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290437    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290440    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290442    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290445    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290449    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290452    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290454    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290457    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:59:02.293255 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290459    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290462    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290464    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290466    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290469    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290471    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290474    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290476    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290479    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290482    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290485    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290487    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290490    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290493    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290495    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290498    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290500    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290503    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290505    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290508    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:59:02.293730 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290510    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290513    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290515    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290878    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290883    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290886    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290889    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290893    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290895    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290898    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290901    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290903    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290906    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290909    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290911    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290914    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290916    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290919    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290922    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290924    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:59:02.294238 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290927    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290930    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290932    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290935    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290938    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290941    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290943    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290946    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290948    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290951    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290953    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290956    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290958    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290960    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290963    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290965    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290968    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290970    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290973    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290975    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:59:02.294748 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290978    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290981    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290983    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290986    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290988    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290991    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290993    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290996    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.290999    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291002    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291004    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291006    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291010    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291013    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291015    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291019    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291023    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291026    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291029    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:59:02.295226 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291032    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291035    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291038    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291040    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291043    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291046    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291049    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291051    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291054    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291056    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291059    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291061    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291064    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291066    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291069    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291072    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291074    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291077    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291079    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:59:02.295731 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291083    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291087    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291090    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291094    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291097    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291100    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291103    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291106    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291109    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291111    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291114    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291116    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291119    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291121    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291124    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291126    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291129    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291131    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291134    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291136    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:59:02.296198 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291139    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291141    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291144    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291146    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291217    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291249    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291255    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291259    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291264    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291267    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291271    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291275    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291292    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291295    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291299    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291302    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291305    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291308    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291311    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291314    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291317    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291320    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291325    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291328    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 07 08:59:02.296698 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291331    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291334    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291337    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291341    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291344    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291347    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291350    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291353    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291357    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291360    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291363    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291366    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291370    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291373    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291376    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291379    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291382    2563 flags.go:64] FLAG: --enable-server="true"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291387    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291392    2563 flags.go:64] FLAG: --event-burst="100"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291395    2563 flags.go:64] FLAG: --event-qps="50"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291398    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291401    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291403    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291407    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 07 08:59:02.297313 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291410    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291412    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291415    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291418    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291421    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291424    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291427    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291430    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291433    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291436    2563 flags.go:64] FLAG: --feature-gates=""
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291440    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291443    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291445    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291448    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291451    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291454    2563 flags.go:64] FLAG: --help="false"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291457    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291460    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291463    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291466    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291469    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291473    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291476    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 07 08:59:02.297937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291478    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291481    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291485    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291489    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291492    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291495    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291498    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291501    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291504    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291506    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291509    2563 flags.go:64] FLAG: --lock-file=""
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291512    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291515    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291518    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291523    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291526    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291529    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291531    2563 flags.go:64] FLAG: --logging-format="text"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291534    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291538    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291540    2563 flags.go:64] FLAG: --manifest-url=""
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291543    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291547    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291550    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291554    2563 flags.go:64] FLAG: --max-pods="110"
Sep 07 08:59:02.298559 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291557    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291559    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291562    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291565    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291568    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291571    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291573    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291581    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291584    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291588    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291591    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291594    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291599    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291602    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291605    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291608    2563 flags.go:64] FLAG: --port="10250"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291611    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291614    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d569f7a0bc53fcd3"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291617    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291620    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291623    2563 flags.go:64] FLAG: --register-node="true"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291626    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291629    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291632    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 07 08:59:02.299170 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291635    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291638    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291641    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291644    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291647    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291650    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291653    2563 flags.go:64] FLAG: --runonce="false"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291656    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291659    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291661    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291664    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291667    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291670    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291673    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291676    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291679    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291682    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291685    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291689    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291692    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291695    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291698    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291703    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291706    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291709    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 07 08:59:02.299778 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291713    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291716    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291719    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291722    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291725    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291727    2563 flags.go:64] FLAG: --v="2"
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291731    2563 flags.go:64] FLAG: --version="false"
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291735    2563 flags.go:64] FLAG: --vmodule=""
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291738    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.291742    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291863    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291867    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291870    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291873    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291877    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291880    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291882    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291885    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291888    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291890    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291893    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:59:02.300399 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291895    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291898    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291900    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291903    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291907    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291910    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291914    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291918    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291921    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291924    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291927    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291930    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291932    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291935    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291938    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291940    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291943    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291946    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291948    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291951    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:59:02.300957 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291953    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291956    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291959    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291961    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291964    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291966    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291969    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291971    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291974    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291977    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291981    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291985    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291988    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291990    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291993    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291996    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.291999    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292002    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292005    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:59:02.301459 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292007    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292010    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292013    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292015    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292018    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292021    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292023    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292026    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292028    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292031    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292033    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292043    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292046    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292048    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292051    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292054    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292056    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292059    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292061    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292064    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:59:02.301933 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292066    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292069    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292071    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292075    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292077    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292080    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292083    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292085    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292088    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292091    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292094    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292097    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292099    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292102    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292105    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292107    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292110    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292112    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292115    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292117    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:59:02.302453 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292120    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292122    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292125    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292127    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292130    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292132    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292135    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292139    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.292143    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.292884    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.299885    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.299899    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.299911    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.299918    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.299980    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:59:02.302942 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.299986    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.299989    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.299993    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.299997    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.299999    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300002    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300005    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300008    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300011    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300017    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300019    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300028    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300031    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300034    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300036    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300039    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300042    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300045    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300048    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300051    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:59:02.303365 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300053    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300056    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300058    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300061    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300064    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300067    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300071    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300074    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300078    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300081    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300084    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300087    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300090    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300093    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300102    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300105    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300108    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300111    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300115    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:59:02.303847 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300118    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300121    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300123    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300126    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300129    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300137    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300140    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300144    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300146    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300149    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300152    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300154    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300157    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300159    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300162    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300164    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300167    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300169    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300172    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300174    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:59:02.304360 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300177    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300180    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300183    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300186    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300189    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300192    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300196    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300200    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300202    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300205    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300207    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300210    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300213    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300215    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300218    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300220    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300223    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300226    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300234    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:59:02.304845 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300237    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300239    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300242    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300244    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300247    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300250    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300252    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300255    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300257    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300260    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300262    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300265    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300267    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300270    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300273    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300275    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300278    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300292    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300295    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:59:02.305326 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300298    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.300303    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300457    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300462    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300465    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300468    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300471    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300474    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300478    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300482    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300484    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300487    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300489    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300499    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:59:02.306004 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300502    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300505    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300508    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300510    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300513    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300515    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300518    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300521    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300523    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300526    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300529    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300531    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300534    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300536    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300539    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300542    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300544    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300547    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300550    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300552    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:59:02.306538 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300554    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300557    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300559    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300562    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300564    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300567    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300570    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300572    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300575    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300577    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300580    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300582    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300591    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300593    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300596    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300598    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300601    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300605    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300608    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:59:02.307028 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300611    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300614    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300617    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300619    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300622    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300624    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300627    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300630    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300633    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300635    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300639    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300642    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300645    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300648    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300650    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300653    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300655    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300658    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300660    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:59:02.307509 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300663    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300665    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300668    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300670    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300673    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300676    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300678    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300686    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300689    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300692    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300695    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300697    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300700    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300703    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300705    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300708    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300711    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300713    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300716    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300719    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:59:02.307980 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300722    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300725    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300727    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300729    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300732    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300735    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300737    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300740    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:02.300742    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.300747    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.300856    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.303020    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.303866    2563 server.go:1019] "Starting client certificate rotation"
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.303984    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 08:59:02.308483 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.304020    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 08:59:02.308827 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.307958    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 08:59:02.311842 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.311826    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 08:59:02.320540 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.320525    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 07 08:59:02.323427 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.323410    2563 log.go:25] "Validated CRI v1 image API"
Sep 07 08:59:02.325355 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.325337    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 07 08:59:02.327900 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.327882    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9ec59cbe-0d17-4219-96c1-6bfc2b8b87d4:/dev/nvme0n1p4 dda3851c-7c1a-4562-9b69-a0197e3a78c2:/dev/nvme0n1p3]
Sep 07 08:59:02.327973 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.327900    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 07 08:59:02.331075 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.331055    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 08:59:02.333391 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.333262    2563 manager.go:216] Machine: {Timestamp:2026-09-07 08:59:02.332085472 +0000 UTC m=+0.301256068 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3043598 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28163fa0893a8310a435d5eae1c52e SystemUUID:ec28163f-a089-3a83-10a4-35d5eae1c52e BootID:c9c6250c-36c4-4ec0-a234-1311484a23fc Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:05:e9:3b:c4:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:05:e9:3b:c4:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:11:71:01:2f:86 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 07 08:59:02.333391 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.333382    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 07 08:59:02.333512 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.333449    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 07 08:59:02.334415 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.334391    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 07 08:59:02.334542 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.334417    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 07 08:59:02.335183 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.335173    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 07 08:59:02.335215 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.335187    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 07 08:59:02.335215 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.335194    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 08:59:02.335786 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.335776    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 08:59:02.337254 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.337244    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 07 08:59:02.341161 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.341150    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 08:59:02.341290 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.341271    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 07 08:59:02.341333 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.341325    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 07 08:59:02.341361 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.341337    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 07 08:59:02.341361 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.341354    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 07 08:59:02.341416 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.341363    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 07 08:59:02.341416 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.341372    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 07 08:59:02.341675 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.341663    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 08:59:02.341724 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.341683    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 08:59:02.343476 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.343459    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 07 08:59:02.346665 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.346643    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 07 08:59:02.346735 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.346697    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 07 08:59:02.346975 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.346962    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 07 08:59:02.347035 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.346981    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 07 08:59:02.347035 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.346990    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 07 08:59:02.347035 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.346999    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 07 08:59:02.347035 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347010    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 07 08:59:02.347035 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347017    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 07 08:59:02.347238 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347234    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 07 08:59:02.347350 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347243    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 07 08:59:02.347350 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347254    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 07 08:59:02.347350 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347263    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 07 08:59:02.347350 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347276    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 07 08:59:02.347350 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347305    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 07 08:59:02.347350 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347327    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 07 08:59:02.347350 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.347335    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 07 08:59:02.350085 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.350063    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-157.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 07 08:59:02.350277 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.350249    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 07 08:59:02.350413 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.350395    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 07 08:59:02.351625 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.351611    2563 server.go:1268] "Started kubelet"
Sep 07 08:59:02.351709 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.351683    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 07 08:59:02.351839 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.351782    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 07 08:59:02.351926 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.351860    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 07 08:59:02.351980 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.351939    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wkp5"
Sep 07 08:59:02.352414 ip-10-0-138-157 systemd[1]: Started Kubernetes Kubelet.
Sep 07 08:59:02.352805 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.352790    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 07 08:59:02.353358 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.353339    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 07 08:59:02.356779 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.355988    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-157.ec2.internal.18d2fe3d8e318aa6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-157.ec2.internal,UID:ip-10-0-138-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-157.ec2.internal,},FirstTimestamp:2026-09-07 08:59:02.351579814 +0000 UTC m=+0.320750412,LastTimestamp:2026-09-07 08:59:02.351579814 +0000 UTC m=+0.320750412,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-157.ec2.internal,}"
Sep 07 08:59:02.357114 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.357088    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 07 08:59:02.358064 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.358049    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 07 08:59:02.358134 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.358071    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 07 08:59:02.358186 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.358139    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:02.358772 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.358756    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 07 08:59:02.358772 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.358756    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 07 08:59:02.358880 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.358780    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 07 08:59:02.358880 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.358830    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 07 08:59:02.358880 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.358837    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 07 08:59:02.359219 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.359202    2563 factory.go:55] Registering systemd factory
Sep 07 08:59:02.359322 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.359242    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 07 08:59:02.359433 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.359421    2563 factory.go:153] Registering CRI-O factory
Sep 07 08:59:02.359466 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.359436    2563 factory.go:223] Registration of the crio container factory successfully
Sep 07 08:59:02.359493 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.359486    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 07 08:59:02.359521 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.359505    2563 factory.go:103] Registering Raw factory
Sep 07 08:59:02.359521 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.359519    2563 manager.go:1196] Started watching for new ooms in manager
Sep 07 08:59:02.359965 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.359950    2563 manager.go:349] Starting recovery of all containers
Sep 07 08:59:02.360508 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.360487    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wkp5"
Sep 07 08:59:02.365810 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.365707    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 07 08:59:02.367996 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.367979    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:59:02.371034 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.371014    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-157.ec2.internal\" not found" node="ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.372293 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.372070    2563 manager.go:354] Recovery completed
Sep 07 08:59:02.378666 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.378654    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:59:02.380497 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.380483    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:59:02.380589 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.380512    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:59:02.380589 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.380526    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:59:02.381126 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.381113    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 07 08:59:02.381126 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.381125    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 07 08:59:02.381257 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.381139    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 08:59:02.381257 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.381196    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:59:02.381978 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.381965    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:59:02.382057 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.381988    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:59:02.382057 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.382001    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:59:02.382310 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.382271    2563 policy_none.go:49] "None policy: Start"
Sep 07 08:59:02.382310 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.382307    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 07 08:59:02.382385 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.382319    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 07 08:59:02.383242 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.383232    2563 policy_none.go:47] "Start"
Sep 07 08:59:02.404839 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.404825    2563 manager.go:337] "Starting Device Plugin manager"
Sep 07 08:59:02.404913 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.404856    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 07 08:59:02.404913 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.404864    2563 server.go:87] "Starting device plugin registration server"
Sep 07 08:59:02.405139 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.405127    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 07 08:59:02.405187 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.405142    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 07 08:59:02.405340 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.405295    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 07 08:59:02.405397 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.405365    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 07 08:59:02.405397 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.405374    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 07 08:59:02.405881 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.405861    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 07 08:59:02.405932 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.405913    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:02.458088 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.458064    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 07 08:59:02.459124 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.458105    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 07 08:59:02.459124 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.458123    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 07 08:59:02.459124 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.458154    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 07 08:59:02.460379 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.460363    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:59:02.505259 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.505234    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:59:02.505983 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.505963    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:59:02.506058 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.505993    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:59:02.506058 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.506003    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:59:02.506058 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.506022    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.511937 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.511923    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.511994 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.511941    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-157.ec2.internal\": node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:02.528377 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.528358    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:02.558620 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.558563    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal"]
Sep 07 08:59:02.558620 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.558622    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:59:02.559807 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.559791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5f89e9b94e651467fe18baa7d2e148-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal\" (UID: \"3f5f89e9b94e651467fe18baa7d2e148\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.559888 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.559814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5f89e9b94e651467fe18baa7d2e148-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal\" (UID: \"3f5f89e9b94e651467fe18baa7d2e148\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.559992 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.559979    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:59:02.560037 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.560002    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:59:02.560037 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.560011    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:59:02.561061 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.561049    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:59:02.561189 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.561174    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.561242 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.561202    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:59:02.561716 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.561693    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:59:02.561786 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.561724    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:59:02.561786 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.561734    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:59:02.561786 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.561705    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:59:02.561890 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.561798    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:59:02.561890 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.561823    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:59:02.562885 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.562869    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.562929 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.562901    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:59:02.563599 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.563581    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:59:02.563674 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.563606    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:59:02.563674 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.563620    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:59:02.577818 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.577797    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-157.ec2.internal\" not found" node="ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.581869 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.581852    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-157.ec2.internal\" not found" node="ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.628652 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.628634    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:02.660696 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.660679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5f89e9b94e651467fe18baa7d2e148-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal\" (UID: \"3f5f89e9b94e651467fe18baa7d2e148\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.660765 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.660705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34b1847e43a3aeca49fcecd34c67497c-config\") pod \"kube-apiserver-proxy-ip-10-0-138-157.ec2.internal\" (UID: \"34b1847e43a3aeca49fcecd34c67497c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.660765 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.660722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5f89e9b94e651467fe18baa7d2e148-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal\" (UID: \"3f5f89e9b94e651467fe18baa7d2e148\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.660765 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.660746    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5f89e9b94e651467fe18baa7d2e148-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal\" (UID: \"3f5f89e9b94e651467fe18baa7d2e148\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.660884 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.660779    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5f89e9b94e651467fe18baa7d2e148-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal\" (UID: \"3f5f89e9b94e651467fe18baa7d2e148\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.728895 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.728873    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:02.761226 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.761202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34b1847e43a3aeca49fcecd34c67497c-config\") pod \"kube-apiserver-proxy-ip-10-0-138-157.ec2.internal\" (UID: \"34b1847e43a3aeca49fcecd34c67497c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.761226 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.761200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34b1847e43a3aeca49fcecd34c67497c-config\") pod \"kube-apiserver-proxy-ip-10-0-138-157.ec2.internal\" (UID: \"34b1847e43a3aeca49fcecd34c67497c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.829518 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.829458    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:02.881134 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.881112    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.885436 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:02.885356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:02.929689 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:02.929666    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:03.030127 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:03.030102    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:03.130626 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:03.130566    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:03.231200 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:03.231178    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:03.303711 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.303686    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 07 08:59:03.304078 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.303817    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 08:59:03.304078 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.303845    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 08:59:03.332015 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:03.331994    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-157.ec2.internal\" not found"
Sep 07 08:59:03.363245 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.363213    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 08:54:03 +0000 UTC" deadline="2028-05-17 07:14:42.864695744 +0000 UTC"
Sep 07 08:59:03.363245 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.363243    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14830h15m39.501456007s"
Sep 07 08:59:03.395219 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.395154    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:59:03.425446 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.425427    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:59:03.458238 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.458215    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:03.468660 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.468645    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 08:59:03.469728 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.469715    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal"
Sep 07 08:59:03.477838 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.477824    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 08:59:03.501903 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:03.501866    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5f89e9b94e651467fe18baa7d2e148.slice/crio-5000a5e1250048c1f686e47eb7fe8ef25f3fa4af03a1e176cd9f9845a732a04a WatchSource:0}: Error finding container 5000a5e1250048c1f686e47eb7fe8ef25f3fa4af03a1e176cd9f9845a732a04a: Status 404 returned error can't find the container with id 5000a5e1250048c1f686e47eb7fe8ef25f3fa4af03a1e176cd9f9845a732a04a
Sep 07 08:59:03.502434 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:03.502411    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b1847e43a3aeca49fcecd34c67497c.slice/crio-cea5c0994c66a559215d23d3f54e8b7d77c34788a64c57af9d3125582efd2dbb WatchSource:0}: Error finding container cea5c0994c66a559215d23d3f54e8b7d77c34788a64c57af9d3125582efd2dbb: Status 404 returned error can't find the container with id cea5c0994c66a559215d23d3f54e8b7d77c34788a64c57af9d3125582efd2dbb
Sep 07 08:59:03.507870 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:03.507856    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 08:59:04.342443 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.342410    2563 apiserver.go:52] "Watching apiserver"
Sep 07 08:59:04.350130 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.349828    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:59:04.350130 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.349865    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:59:04.353677 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.353650    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 07 08:59:04.354081 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.354052    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-vvtcs","kube-system/global-pull-secret-syncer-xpfh7","kube-system/konnectivity-agent-m6gv9","openshift-cluster-node-tuning-operator/tuned-tbwx8","openshift-image-registry/node-ca-bw5zk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal","openshift-multus/multus-tg9lx","openshift-multus/network-metrics-daemon-bmllm","openshift-network-operator/iptables-alerter-kps6t","kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm","openshift-dns/node-resolver-fkt8f","openshift-multus/multus-additional-cni-plugins-fdspn","openshift-network-diagnostics/network-check-target-f7bgm"]
Sep 07 08:59:04.356935 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.356911    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.359369 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.359349    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 07 08:59:04.360107 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.359587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.360107 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.359735    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 07 08:59:04.360107 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.359974    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 07 08:59:04.360107 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.360038    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5plsn\""
Sep 07 08:59:04.360395 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.360147    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 07 08:59:04.360395 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.360184    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 07 08:59:04.361831 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.361814    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 07 08:59:04.362075 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.362055    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:04.362981 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.362959    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 07 08:59:04.363164 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.362964    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r2ppj\""
Sep 07 08:59:04.364533 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.364505    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.365003 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.364697    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w4g7r\""
Sep 07 08:59:04.365003 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.364793    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 07 08:59:04.365003 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.364948    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 07 08:59:04.367199 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.367177    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 07 08:59:04.367714 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.367488    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 07 08:59:04.367714 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.367488    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kssgh\""
Sep 07 08:59:04.367714 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.367693    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.369727 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.369707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:04.369827 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.369783    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:04.370492 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.370470    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 07 08:59:04.370699 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.370681    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 07 08:59:04.370960 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.370935    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fbrg4\""
Sep 07 08:59:04.371066 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-cnibin\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371149 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53d15ca2-406b-450f-9fc2-196927235f32-cni-binary-copy\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371149 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b06f9c26-d0eb-4e4d-8424-51087b91fa18-cni-binary-copy\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.371149 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371128    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 07 08:59:04.371301 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2767bb9a-644e-4a87-999c-a90f8916437b-agent-certs\") pod \"konnectivity-agent-m6gv9\" (UID: \"2767bb9a-644e-4a87-999c-a90f8916437b\") " pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:04.371381 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/53d15ca2-406b-450f-9fc2-196927235f32-multus-daemon-config\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371442 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371356    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94vvs\" (UniqueName: \"kubernetes.io/projected/53d15ca2-406b-450f-9fc2-196927235f32-kube-api-access-94vvs\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371496 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371448    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b06f9c26-d0eb-4e4d-8424-51087b91fa18-cnibin\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.371496 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b06f9c26-d0eb-4e4d-8424-51087b91fa18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.371604 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-var-lib-kubelet\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371604 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-multus-socket-dir-parent\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371604 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-run-k8s-cni-cncf-io\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371739 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-multus-conf-dir\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371739 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b06f9c26-d0eb-4e4d-8424-51087b91fa18-system-cni-dir\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.371739 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371697    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b06f9c26-d0eb-4e4d-8424-51087b91fa18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.371878 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-run-netns\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371878 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-var-lib-cni-multus\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.371878 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371832    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-hostroot\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.372015 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-run-multus-certs\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.372015 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-etc-kubernetes\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.372015 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.371942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b06f9c26-d0eb-4e4d-8424-51087b91fa18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.372473 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.372446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.372814 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.372526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndq8m\" (UniqueName: \"kubernetes.io/projected/b06f9c26-d0eb-4e4d-8424-51087b91fa18-kube-api-access-ndq8m\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.372985 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.372965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2767bb9a-644e-4a87-999c-a90f8916437b-konnectivity-ca\") pod \"konnectivity-agent-m6gv9\" (UID: \"2767bb9a-644e-4a87-999c-a90f8916437b\") " pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:04.373690 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.373036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-var-lib-cni-bin\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.373690 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.373065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-system-cni-dir\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.373690 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.373140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-multus-cni-dir\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.373690 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.373422    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 08:59:04.374839 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.374815    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.376066 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.375861    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 07 08:59:04.376066 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.375913    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 07 08:59:04.376066 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.375922    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9m8p9\""
Sep 07 08:59:04.376390 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.376350    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 07 08:59:04.377499 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.377365    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:04.377499 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.377443    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:04.377947 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.377778    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 07 08:59:04.378064 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.378045    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r9b7x\""
Sep 07 08:59:04.378145 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.378049    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 07 08:59:04.378211 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.378177    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 07 08:59:04.379427 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.379091    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 07 08:59:04.379427 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.379171    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 07 08:59:04.379427 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.379216    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 07 08:59:04.380589 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.380180    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:04.380589 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.380522    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:04.385159 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.385033    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.387706 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.387684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.387942 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.387920    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 07 08:59:04.388195 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.388176    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 07 08:59:04.388302 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.388277    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 07 08:59:04.388439 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.388399    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mxdzv\""
Sep 07 08:59:04.390206 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.390187    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 07 08:59:04.390885 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.390862    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 07 08:59:04.390983 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.390920    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hzjn7\""
Sep 07 08:59:04.399712 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.399685    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2sr2"
Sep 07 08:59:04.407605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.407583    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2sr2"
Sep 07 08:59:04.460484 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.460462    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 07 08:59:04.462974 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.462931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal" event={"ID":"34b1847e43a3aeca49fcecd34c67497c","Type":"ContainerStarted","Data":"cea5c0994c66a559215d23d3f54e8b7d77c34788a64c57af9d3125582efd2dbb"}
Sep 07 08:59:04.464332 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.464305    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal" event={"ID":"3f5f89e9b94e651467fe18baa7d2e148","Type":"ContainerStarted","Data":"5000a5e1250048c1f686e47eb7fe8ef25f3fa4af03a1e176cd9f9845a732a04a"}
Sep 07 08:59:04.474174 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-systemd-units\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.474312 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-lib-modules\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.474312 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474218    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-cnibin\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.474312 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474243    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c8d36ac-5cab-446f-8e12-19f48927540c-host\") pod \"node-ca-bw5zk\" (UID: \"2c8d36ac-5cab-446f-8e12-19f48927540c\") " pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.474487 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-cni-netd\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.474487 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-cnibin\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.474487 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/762acdac-34ca-469a-87e9-e945fb096c34-ovnkube-script-lib\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.474487 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-var-lib-kubelet\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.474487 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3397570-2652-435f-87ba-722b12bba06e-tmp\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.474733 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/53d15ca2-406b-450f-9fc2-196927235f32-multus-daemon-config\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.474733 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b06f9c26-d0eb-4e4d-8424-51087b91fa18-cnibin\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.474733 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jr6r\" (UniqueName: \"kubernetes.io/projected/2c8d36ac-5cab-446f-8e12-19f48927540c-kube-api-access-2jr6r\") pod \"node-ca-bw5zk\" (UID: \"2c8d36ac-5cab-446f-8e12-19f48927540c\") " pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.474733 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/762acdac-34ca-469a-87e9-e945fb096c34-ovn-node-metrics-cert\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.474733 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474641    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:04.474733 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b06f9c26-d0eb-4e4d-8424-51087b91fa18-cnibin\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.474733 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pms6l\" (UniqueName: \"kubernetes.io/projected/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-kube-api-access-pms6l\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:04.474733 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-modprobe-d\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.474733 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-run-openvswitch\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-host\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-kubelet-config\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-var-lib-cni-multus\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-hostroot\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-var-lib-cni-multus\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-hostroot\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-kubelet\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-run-netns\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.474963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-sysctl-d\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-sys\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-var-lib-cni-bin\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-multus-cni-dir\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53d15ca2-406b-450f-9fc2-196927235f32-cni-binary-copy\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2767bb9a-644e-4a87-999c-a90f8916437b-agent-certs\") pod \"konnectivity-agent-m6gv9\" (UID: \"2767bb9a-644e-4a87-999c-a90f8916437b\") " pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475112    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/53d15ca2-406b-450f-9fc2-196927235f32-multus-daemon-config\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-var-lib-cni-bin\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.475196 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-kubernetes\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94vvs\" (UniqueName: \"kubernetes.io/projected/53d15ca2-406b-450f-9fc2-196927235f32-kube-api-access-94vvs\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475239    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475175    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-multus-cni-dir\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-var-lib-kubelet\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b06f9c26-d0eb-4e4d-8424-51087b91fa18-system-cni-dir\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475368    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4a9f8a1c-0d68-43ba-9d24-51290db60316-iptables-alerter-script\") pod \"iptables-alerter-kps6t\" (UID: \"4a9f8a1c-0d68-43ba-9d24-51290db60316\") " pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475398    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-run-ovn\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-var-lib-kubelet\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475424    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b06f9c26-d0eb-4e4d-8424-51087b91fa18-system-cni-dir\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-run-netns\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-run-multus-certs\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475486    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-run-netns\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b06f9c26-d0eb-4e4d-8424-51087b91fa18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndq8m\" (UniqueName: \"kubernetes.io/projected/b06f9c26-d0eb-4e4d-8424-51087b91fa18-kube-api-access-ndq8m\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475549    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-run-multus-certs\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2767bb9a-644e-4a87-999c-a90f8916437b-konnectivity-ca\") pod \"konnectivity-agent-m6gv9\" (UID: \"2767bb9a-644e-4a87-999c-a90f8916437b\") " pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:04.476113 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4jfr\" (UniqueName: \"kubernetes.io/projected/4a9f8a1c-0d68-43ba-9d24-51290db60316-kube-api-access-h4jfr\") pod \"iptables-alerter-kps6t\" (UID: \"4a9f8a1c-0d68-43ba-9d24-51290db60316\") " pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-var-lib-openvswitch\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-cni-bin\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475684    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53d15ca2-406b-450f-9fc2-196927235f32-cni-binary-copy\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475686    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b06f9c26-d0eb-4e4d-8424-51087b91fa18-cni-binary-copy\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475778    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-etc-openvswitch\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4thvv\" (UniqueName: \"kubernetes.io/projected/762acdac-34ca-469a-87e9-e945fb096c34-kube-api-access-4thvv\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-systemd\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475894    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3397570-2652-435f-87ba-722b12bba06e-etc-tuned\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83680078-ce21-45b5-b66c-c4211b91f06b-tmp-dir\") pod \"node-resolver-fkt8f\" (UID: \"83680078-ce21-45b5-b66c-c4211b91f06b\") " pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.475979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8cpz\" (UniqueName: \"kubernetes.io/projected/83680078-ce21-45b5-b66c-c4211b91f06b-kube-api-access-l8cpz\") pod \"node-resolver-fkt8f\" (UID: \"83680078-ce21-45b5-b66c-c4211b91f06b\") " pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a9f8a1c-0d68-43ba-9d24-51290db60316-host-slash\") pod \"iptables-alerter-kps6t\" (UID: \"4a9f8a1c-0d68-43ba-9d24-51290db60316\") " pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-run-systemd\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476058    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b06f9c26-d0eb-4e4d-8424-51087b91fa18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrwhf\" (UniqueName: \"kubernetes.io/projected/43e2c740-a35b-4b80-b15a-82e83b47e1db-kube-api-access-mrwhf\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.476955 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-run-k8s-cni-cncf-io\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476152    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2767bb9a-644e-4a87-999c-a90f8916437b-konnectivity-ca\") pod \"konnectivity-agent-m6gv9\" (UID: \"2767bb9a-644e-4a87-999c-a90f8916437b\") " pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b06f9c26-d0eb-4e4d-8424-51087b91fa18-cni-binary-copy\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-host-run-k8s-cni-cncf-io\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b06f9c26-d0eb-4e4d-8424-51087b91fa18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-log-socket\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/762acdac-34ca-469a-87e9-e945fb096c34-env-overrides\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-sysconfig\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-sysctl-conf\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-socket-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-etc-kubernetes\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476508    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-slash\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-run\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-dbus\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-registration-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.477605 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-sys-fs\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-system-cni-dir\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b06f9c26-d0eb-4e4d-8424-51087b91fa18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-etc-kubernetes\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c8d36ac-5cab-446f-8e12-19f48927540c-serviceca\") pod \"node-ca-bw5zk\" (UID: \"2c8d36ac-5cab-446f-8e12-19f48927540c\") " pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-run-ovn-kubernetes\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476943    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-system-cni-dir\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476952    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83680078-ce21-45b5-b66c-c4211b91f06b-hosts-file\") pod \"node-resolver-fkt8f\" (UID: \"83680078-ce21-45b5-b66c-c4211b91f06b\") " pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.476993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b06f9c26-d0eb-4e4d-8424-51087b91fa18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pfq6\" (UniqueName: \"kubernetes.io/projected/f3397570-2652-435f-87ba-722b12bba06e-kube-api-access-2pfq6\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-device-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b06f9c26-d0eb-4e4d-8424-51087b91fa18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-multus-socket-dir-parent\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-multus-conf-dir\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-node-log\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-multus-socket-dir-parent\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.478474 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/762acdac-34ca-469a-87e9-e945fb096c34-ovnkube-config\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.479391 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53d15ca2-406b-450f-9fc2-196927235f32-multus-conf-dir\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.479391 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.477392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-etc-selinux\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.481439 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.481397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2767bb9a-644e-4a87-999c-a90f8916437b-agent-certs\") pod \"konnectivity-agent-m6gv9\" (UID: \"2767bb9a-644e-4a87-999c-a90f8916437b\") " pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:04.483922 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.483899    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndq8m\" (UniqueName: \"kubernetes.io/projected/b06f9c26-d0eb-4e4d-8424-51087b91fa18-kube-api-access-ndq8m\") pod \"multus-additional-cni-plugins-fdspn\" (UID: \"b06f9c26-d0eb-4e4d-8424-51087b91fa18\") " pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.485129 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.485108    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94vvs\" (UniqueName: \"kubernetes.io/projected/53d15ca2-406b-450f-9fc2-196927235f32-kube-api-access-94vvs\") pod \"multus-tg9lx\" (UID: \"53d15ca2-406b-450f-9fc2-196927235f32\") " pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.578295 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-kubernetes\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.578439 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:04.578439 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4a9f8a1c-0d68-43ba-9d24-51290db60316-iptables-alerter-script\") pod \"iptables-alerter-kps6t\" (UID: \"4a9f8a1c-0d68-43ba-9d24-51290db60316\") " pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.578439 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-run-ovn\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.578439 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4jfr\" (UniqueName: \"kubernetes.io/projected/4a9f8a1c-0d68-43ba-9d24-51290db60316-kube-api-access-h4jfr\") pod \"iptables-alerter-kps6t\" (UID: \"4a9f8a1c-0d68-43ba-9d24-51290db60316\") " pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.578439 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-kubernetes\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.578439 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-var-lib-openvswitch\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.578518    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-cni-bin\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-etc-openvswitch\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578591    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-var-lib-openvswitch\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-etc-openvswitch\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.578603    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret podName:da4c2b44-97ef-47ec-812b-6f3bb1b5c536 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:05.078579479 +0000 UTC m=+3.047750065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret") pod "global-pull-secret-syncer-xpfh7" (UID: "da4c2b44-97ef-47ec-812b-6f3bb1b5c536") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-run-ovn\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4thvv\" (UniqueName: \"kubernetes.io/projected/762acdac-34ca-469a-87e9-e945fb096c34-kube-api-access-4thvv\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-systemd\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-cni-bin\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.578726 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3397570-2652-435f-87ba-722b12bba06e-etc-tuned\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578875    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578886    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-systemd\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83680078-ce21-45b5-b66c-c4211b91f06b-tmp-dir\") pod \"node-resolver-fkt8f\" (UID: \"83680078-ce21-45b5-b66c-c4211b91f06b\") " pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8cpz\" (UniqueName: \"kubernetes.io/projected/83680078-ce21-45b5-b66c-c4211b91f06b-kube-api-access-l8cpz\") pod \"node-resolver-fkt8f\" (UID: \"83680078-ce21-45b5-b66c-c4211b91f06b\") " pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.578952    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579007    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4a9f8a1c-0d68-43ba-9d24-51290db60316-iptables-alerter-script\") pod \"iptables-alerter-kps6t\" (UID: \"4a9f8a1c-0d68-43ba-9d24-51290db60316\") " pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a9f8a1c-0d68-43ba-9d24-51290db60316-host-slash\") pod \"iptables-alerter-kps6t\" (UID: \"4a9f8a1c-0d68-43ba-9d24-51290db60316\") " pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-run-systemd\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrwhf\" (UniqueName: \"kubernetes.io/projected/43e2c740-a35b-4b80-b15a-82e83b47e1db-kube-api-access-mrwhf\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a9f8a1c-0d68-43ba-9d24-51290db60316-host-slash\") pod \"iptables-alerter-kps6t\" (UID: \"4a9f8a1c-0d68-43ba-9d24-51290db60316\") " pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-log-socket\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-run-systemd\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/762acdac-34ca-469a-87e9-e945fb096c34-env-overrides\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.579202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579201    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-log-socket\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-sysconfig\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-sysctl-conf\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-socket-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83680078-ce21-45b5-b66c-c4211b91f06b-tmp-dir\") pod \"node-resolver-fkt8f\" (UID: \"83680078-ce21-45b5-b66c-c4211b91f06b\") " pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-slash\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-run\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579380    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-sysconfig\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-dbus\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579409    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-sysctl-conf\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-registration-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-sys-fs\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-registration-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c8d36ac-5cab-446f-8e12-19f48927540c-serviceca\") pod \"node-ca-bw5zk\" (UID: \"2c8d36ac-5cab-446f-8e12-19f48927540c\") " pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-slash\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-run-ovn-kubernetes\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.579683 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83680078-ce21-45b5-b66c-c4211b91f06b-hosts-file\") pod \"node-resolver-fkt8f\" (UID: \"83680078-ce21-45b5-b66c-c4211b91f06b\") " pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-run\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579597    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/762acdac-34ca-469a-87e9-e945fb096c34-env-overrides\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pfq6\" (UniqueName: \"kubernetes.io/projected/f3397570-2652-435f-87ba-722b12bba06e-kube-api-access-2pfq6\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-dbus\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579615    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-sys-fs\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579486    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-socket-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579644    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-device-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83680078-ce21-45b5-b66c-c4211b91f06b-hosts-file\") pod \"node-resolver-fkt8f\" (UID: \"83680078-ce21-45b5-b66c-c4211b91f06b\") " pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579690    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-device-dir\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-node-log\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-node-log\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/762acdac-34ca-469a-87e9-e945fb096c34-ovnkube-config\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-etc-selinux\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-run-ovn-kubernetes\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.580204 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-systemd-units\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-lib-modules\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c8d36ac-5cab-446f-8e12-19f48927540c-host\") pod \"node-ca-bw5zk\" (UID: \"2c8d36ac-5cab-446f-8e12-19f48927540c\") " pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-cni-netd\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-systemd-units\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/762acdac-34ca-469a-87e9-e945fb096c34-ovnkube-script-lib\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579911    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c8d36ac-5cab-446f-8e12-19f48927540c-serviceca\") pod \"node-ca-bw5zk\" (UID: \"2c8d36ac-5cab-446f-8e12-19f48927540c\") " pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/43e2c740-a35b-4b80-b15a-82e83b47e1db-etc-selinux\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c8d36ac-5cab-446f-8e12-19f48927540c-host\") pod \"node-ca-bw5zk\" (UID: \"2c8d36ac-5cab-446f-8e12-19f48927540c\") " pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579942    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-var-lib-kubelet\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-cni-netd\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.579991    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-var-lib-kubelet\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3397570-2652-435f-87ba-722b12bba06e-tmp\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jr6r\" (UniqueName: \"kubernetes.io/projected/2c8d36ac-5cab-446f-8e12-19f48927540c-kube-api-access-2jr6r\") pod \"node-ca-bw5zk\" (UID: \"2c8d36ac-5cab-446f-8e12-19f48927540c\") " pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/762acdac-34ca-469a-87e9-e945fb096c34-ovn-node-metrics-cert\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-lib-modules\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pms6l\" (UniqueName: \"kubernetes.io/projected/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-kube-api-access-pms6l\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:04.581028 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-modprobe-d\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.580246    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.580293    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs podName:79f55277-100d-44e8-b8e4-a1ca8c9ed7e5 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:05.080277506 +0000 UTC m=+3.049448096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs") pod "network-metrics-daemon-bmllm" (UID: "79f55277-100d-44e8-b8e4-a1ca8c9ed7e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-modprobe-d\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-run-openvswitch\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-host\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-kubelet-config\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-kubelet\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-run-netns\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-sysctl-d\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-sys\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580543    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/762acdac-34ca-469a-87e9-e945fb096c34-ovnkube-script-lib\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/762acdac-34ca-469a-87e9-e945fb096c34-ovnkube-config\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-kubelet-config\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-sys\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-run-openvswitch\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-host\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.581903 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-kubelet\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.582658 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3397570-2652-435f-87ba-722b12bba06e-etc-sysctl-d\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.582658 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.580665    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/762acdac-34ca-469a-87e9-e945fb096c34-host-run-netns\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.582658 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.581728    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3397570-2652-435f-87ba-722b12bba06e-etc-tuned\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.583346 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.583323    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3397570-2652-435f-87ba-722b12bba06e-tmp\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.583456 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.583437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/762acdac-34ca-469a-87e9-e945fb096c34-ovn-node-metrics-cert\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.587809 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.587785    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:59:04.587957 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.587814    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:59:04.587957 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.587829    2563 projected.go:196] Error preparing data for projected volume kube-api-access-v7zwd for pod openshift-network-diagnostics/network-check-target-f7bgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:04.587957 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:04.587903    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd podName:6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:05.087886047 +0000 UTC m=+3.057056653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v7zwd" (UniqueName: "kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd") pod "network-check-target-f7bgm" (UID: "6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:04.590433 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.590397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4thvv\" (UniqueName: \"kubernetes.io/projected/762acdac-34ca-469a-87e9-e945fb096c34-kube-api-access-4thvv\") pod \"ovnkube-node-vvtcs\" (UID: \"762acdac-34ca-469a-87e9-e945fb096c34\") " pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.591667 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.591643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jr6r\" (UniqueName: \"kubernetes.io/projected/2c8d36ac-5cab-446f-8e12-19f48927540c-kube-api-access-2jr6r\") pod \"node-ca-bw5zk\" (UID: \"2c8d36ac-5cab-446f-8e12-19f48927540c\") " pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.592013 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.591989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8cpz\" (UniqueName: \"kubernetes.io/projected/83680078-ce21-45b5-b66c-c4211b91f06b-kube-api-access-l8cpz\") pod \"node-resolver-fkt8f\" (UID: \"83680078-ce21-45b5-b66c-c4211b91f06b\") " pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:04.592541 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.592485    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pms6l\" (UniqueName: \"kubernetes.io/projected/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-kube-api-access-pms6l\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:04.592654 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.592622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pfq6\" (UniqueName: \"kubernetes.io/projected/f3397570-2652-435f-87ba-722b12bba06e-kube-api-access-2pfq6\") pod \"tuned-tbwx8\" (UID: \"f3397570-2652-435f-87ba-722b12bba06e\") " pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.593119 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.593099    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrwhf\" (UniqueName: \"kubernetes.io/projected/43e2c740-a35b-4b80-b15a-82e83b47e1db-kube-api-access-mrwhf\") pod \"aws-ebs-csi-driver-node-vk2sm\" (UID: \"43e2c740-a35b-4b80-b15a-82e83b47e1db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.593218 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.593195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4jfr\" (UniqueName: \"kubernetes.io/projected/4a9f8a1c-0d68-43ba-9d24-51290db60316-kube-api-access-h4jfr\") pod \"iptables-alerter-kps6t\" (UID: \"4a9f8a1c-0d68-43ba-9d24-51290db60316\") " pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.670145 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.670120    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tg9lx"
Sep 07 08:59:04.679754 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.679732    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fdspn"
Sep 07 08:59:04.691661 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.691638    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:04.697564 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.697542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tbwx8"
Sep 07 08:59:04.707432 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.707414    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bw5zk"
Sep 07 08:59:04.717911 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.717893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kps6t"
Sep 07 08:59:04.726735 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.726718    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:04.732913 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.732896    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm"
Sep 07 08:59:04.739614 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:04.739593    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fkt8f"
Sep 07 08:59:05.084780 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:05.084696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:05.084780 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:05.084749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:05.085017 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:05.084872    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:05.085017 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:05.084877    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:05.085017 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:05.084930    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret podName:da4c2b44-97ef-47ec-812b-6f3bb1b5c536 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:06.084915224 +0000 UTC m=+4.054085809 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret") pod "global-pull-secret-syncer-xpfh7" (UID: "da4c2b44-97ef-47ec-812b-6f3bb1b5c536") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:05.085017 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:05.084946    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs podName:79f55277-100d-44e8-b8e4-a1ca8c9ed7e5 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:06.084939897 +0000 UTC m=+4.054110481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs") pod "network-metrics-daemon-bmllm" (UID: "79f55277-100d-44e8-b8e4-a1ca8c9ed7e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:05.186121 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:05.186088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:05.186263 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:05.186211    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:59:05.186263 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:05.186228    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:59:05.186263 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:05.186237    2563 projected.go:196] Error preparing data for projected volume kube-api-access-v7zwd for pod openshift-network-diagnostics/network-check-target-f7bgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:05.186397 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:05.186285    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd podName:6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:06.186272294 +0000 UTC m=+4.155442879 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7zwd" (UniqueName: "kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd") pod "network-check-target-f7bgm" (UID: "6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:05.411858 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:05.411771    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 08:54:05 +0000 UTC" deadline="2028-05-28 01:15:22.58529975 +0000 UTC"
Sep 07 08:59:05.411858 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:05.411807    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15088h16m17.173496335s"
Sep 07 08:59:05.484562 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:05.484332    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3397570_2652_435f_87ba_722b12bba06e.slice/crio-6dac61559eb3244ac213915a0c0d97e2d024f2946e61b214be091d7c9d1a1177 WatchSource:0}: Error finding container 6dac61559eb3244ac213915a0c0d97e2d024f2946e61b214be091d7c9d1a1177: Status 404 returned error can't find the container with id 6dac61559eb3244ac213915a0c0d97e2d024f2946e61b214be091d7c9d1a1177
Sep 07 08:59:05.506405 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:05.506313    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8d36ac_5cab_446f_8e12_19f48927540c.slice/crio-c8abd41ed13dd3e695bb7619d64bc64f89455070827b091713bcad52ceb2767b WatchSource:0}: Error finding container c8abd41ed13dd3e695bb7619d64bc64f89455070827b091713bcad52ceb2767b: Status 404 returned error can't find the container with id c8abd41ed13dd3e695bb7619d64bc64f89455070827b091713bcad52ceb2767b
Sep 07 08:59:05.507551 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:05.507526    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e2c740_a35b_4b80_b15a_82e83b47e1db.slice/crio-d92b5032dca1931c03334de24a73c62056de95210b321f2e49720a456e221cc8 WatchSource:0}: Error finding container d92b5032dca1931c03334de24a73c62056de95210b321f2e49720a456e221cc8: Status 404 returned error can't find the container with id d92b5032dca1931c03334de24a73c62056de95210b321f2e49720a456e221cc8
Sep 07 08:59:05.508876 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:05.508472    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d15ca2_406b_450f_9fc2_196927235f32.slice/crio-2673233a68c5f4ecac718739e8fbb183365a01cfa6bd1acbda4b76afd2305bc3 WatchSource:0}: Error finding container 2673233a68c5f4ecac718739e8fbb183365a01cfa6bd1acbda4b76afd2305bc3: Status 404 returned error can't find the container with id 2673233a68c5f4ecac718739e8fbb183365a01cfa6bd1acbda4b76afd2305bc3
Sep 07 08:59:05.509308 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:05.509262    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83680078_ce21_45b5_b66c_c4211b91f06b.slice/crio-2ebaec3ce4ebb8cd563191eb5307bec87c3247387f3aa8c427f5448cbd232a81 WatchSource:0}: Error finding container 2ebaec3ce4ebb8cd563191eb5307bec87c3247387f3aa8c427f5448cbd232a81: Status 404 returned error can't find the container with id 2ebaec3ce4ebb8cd563191eb5307bec87c3247387f3aa8c427f5448cbd232a81
Sep 07 08:59:05.510265 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:05.510219    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9f8a1c_0d68_43ba_9d24_51290db60316.slice/crio-ccf6602f7a9cd2b98b854262684f4b5f4d78fb62f050dce1b7cbcf927e0a1ea0 WatchSource:0}: Error finding container ccf6602f7a9cd2b98b854262684f4b5f4d78fb62f050dce1b7cbcf927e0a1ea0: Status 404 returned error can't find the container with id ccf6602f7a9cd2b98b854262684f4b5f4d78fb62f050dce1b7cbcf927e0a1ea0
Sep 07 08:59:05.511539 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:05.511516    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762acdac_34ca_469a_87e9_e945fb096c34.slice/crio-9423c4830676f02ef66712f8013d2986a57879a977bd1f822877d7e52c9e3d81 WatchSource:0}: Error finding container 9423c4830676f02ef66712f8013d2986a57879a977bd1f822877d7e52c9e3d81: Status 404 returned error can't find the container with id 9423c4830676f02ef66712f8013d2986a57879a977bd1f822877d7e52c9e3d81
Sep 07 08:59:06.093437 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.093356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:06.093437 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.093416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:06.093698 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.093656    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:06.093761 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.093729    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret podName:da4c2b44-97ef-47ec-812b-6f3bb1b5c536 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:08.093709539 +0000 UTC m=+6.062880129 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret") pod "global-pull-secret-syncer-xpfh7" (UID: "da4c2b44-97ef-47ec-812b-6f3bb1b5c536") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:06.094175 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.094154    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:06.094245 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.094213    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs podName:79f55277-100d-44e8-b8e4-a1ca8c9ed7e5 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:08.094196164 +0000 UTC m=+6.063366768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs") pod "network-metrics-daemon-bmllm" (UID: "79f55277-100d-44e8-b8e4-a1ca8c9ed7e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:06.195068 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.195027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:06.195245 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.195199    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:59:06.195245 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.195220    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:59:06.195245 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.195232    2563 projected.go:196] Error preparing data for projected volume kube-api-access-v7zwd for pod openshift-network-diagnostics/network-check-target-f7bgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:06.195443 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.195290    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd podName:6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:08.19527316 +0000 UTC m=+6.164443750 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7zwd" (UniqueName: "kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd") pod "network-check-target-f7bgm" (UID: "6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:06.412131 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.411983    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 08:54:05 +0000 UTC" deadline="2028-05-10 14:07:29.516016616 +0000 UTC"
Sep 07 08:59:06.412131 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.412047    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14669h8m23.103974311s"
Sep 07 08:59:06.460979 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.460949    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:06.461148 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.461077    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:06.461807 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.461633    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:06.461807 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.461762    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:06.462057 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.461837    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:06.462057 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:06.461950    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:06.506800 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.506135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal" event={"ID":"34b1847e43a3aeca49fcecd34c67497c","Type":"ContainerStarted","Data":"68e6cb2b43c356c9889503dbde2a08ce7cdee45bf2860e893ac75f16f008bbc6"}
Sep 07 08:59:06.517346 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.516990    2563 generic.go:356] "Generic (PLEG): container finished" podID="3f5f89e9b94e651467fe18baa7d2e148" containerID="c0c1b54a000ee55755f62677307a49cf733e38db70c42ac770e36696aeb68aad" exitCode=0
Sep 07 08:59:06.517346 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.517075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal" event={"ID":"3f5f89e9b94e651467fe18baa7d2e148","Type":"ContainerDied","Data":"c0c1b54a000ee55755f62677307a49cf733e38db70c42ac770e36696aeb68aad"}
Sep 07 08:59:06.522701 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.522295    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-157.ec2.internal" podStartSLOduration=2.522280421 podStartE2EDuration="2.522280421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:06.52109764 +0000 UTC m=+4.490268250" watchObservedRunningTime="2026-09-07 08:59:06.522280421 +0000 UTC m=+4.491451029"
Sep 07 08:59:06.524544 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.524515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kps6t" event={"ID":"4a9f8a1c-0d68-43ba-9d24-51290db60316","Type":"ContainerStarted","Data":"ccf6602f7a9cd2b98b854262684f4b5f4d78fb62f050dce1b7cbcf927e0a1ea0"}
Sep 07 08:59:06.539505 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.539475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm" event={"ID":"43e2c740-a35b-4b80-b15a-82e83b47e1db","Type":"ContainerStarted","Data":"d92b5032dca1931c03334de24a73c62056de95210b321f2e49720a456e221cc8"}
Sep 07 08:59:06.541431 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.541405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bw5zk" event={"ID":"2c8d36ac-5cab-446f-8e12-19f48927540c","Type":"ContainerStarted","Data":"c8abd41ed13dd3e695bb7619d64bc64f89455070827b091713bcad52ceb2767b"}
Sep 07 08:59:06.556437 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.556409    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdspn" event={"ID":"b06f9c26-d0eb-4e4d-8424-51087b91fa18","Type":"ContainerStarted","Data":"efd79f33a55237f71a761651870eb38b91ab626ba9eb4d3ed3b0080bcb2f2423"}
Sep 07 08:59:06.571919 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.571888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m6gv9" event={"ID":"2767bb9a-644e-4a87-999c-a90f8916437b","Type":"ContainerStarted","Data":"1c367fb4abb1c25cb7901c6e3cca659f1d10fcdf6c3a52b6e97c7d923de9563c"}
Sep 07 08:59:06.587954 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.587863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" event={"ID":"762acdac-34ca-469a-87e9-e945fb096c34","Type":"ContainerStarted","Data":"9423c4830676f02ef66712f8013d2986a57879a977bd1f822877d7e52c9e3d81"}
Sep 07 08:59:06.597697 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.597663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fkt8f" event={"ID":"83680078-ce21-45b5-b66c-c4211b91f06b","Type":"ContainerStarted","Data":"2ebaec3ce4ebb8cd563191eb5307bec87c3247387f3aa8c427f5448cbd232a81"}
Sep 07 08:59:06.601431 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.601399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tg9lx" event={"ID":"53d15ca2-406b-450f-9fc2-196927235f32","Type":"ContainerStarted","Data":"2673233a68c5f4ecac718739e8fbb183365a01cfa6bd1acbda4b76afd2305bc3"}
Sep 07 08:59:06.603648 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:06.603619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tbwx8" event={"ID":"f3397570-2652-435f-87ba-722b12bba06e","Type":"ContainerStarted","Data":"6dac61559eb3244ac213915a0c0d97e2d024f2946e61b214be091d7c9d1a1177"}
Sep 07 08:59:07.612315 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:07.612248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal" event={"ID":"3f5f89e9b94e651467fe18baa7d2e148","Type":"ContainerStarted","Data":"b3cfd22ff3303f5e22442430c960b23e4b2577720ddbb589f194fa512231ac3c"}
Sep 07 08:59:07.629404 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:07.629329    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-157.ec2.internal" podStartSLOduration=3.629305724 podStartE2EDuration="3.629305724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:07.628785367 +0000 UTC m=+5.597955975" watchObservedRunningTime="2026-09-07 08:59:07.629305724 +0000 UTC m=+5.598476334"
Sep 07 08:59:08.113212 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:08.112368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:08.113212 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:08.112429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:08.113212 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.112590    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:08.113212 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.112652    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret podName:da4c2b44-97ef-47ec-812b-6f3bb1b5c536 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:12.112634331 +0000 UTC m=+10.081804931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret") pod "global-pull-secret-syncer-xpfh7" (UID: "da4c2b44-97ef-47ec-812b-6f3bb1b5c536") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:08.113212 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.113061    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:08.113212 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.113115    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs podName:79f55277-100d-44e8-b8e4-a1ca8c9ed7e5 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:12.113099061 +0000 UTC m=+10.082269656 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs") pod "network-metrics-daemon-bmllm" (UID: "79f55277-100d-44e8-b8e4-a1ca8c9ed7e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:08.213360 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:08.212734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:08.213360 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.212923    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:59:08.213360 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.212952    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:59:08.213360 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.212967    2563 projected.go:196] Error preparing data for projected volume kube-api-access-v7zwd for pod openshift-network-diagnostics/network-check-target-f7bgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:08.213360 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.213023    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd podName:6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:12.2130064 +0000 UTC m=+10.182177000 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7zwd" (UniqueName: "kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd") pod "network-check-target-f7bgm" (UID: "6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:08.460131 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:08.460045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:08.460332 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.460231    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:08.460681 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:08.460651    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:08.460809 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.460781    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:08.460891 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:08.460860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:08.461019 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:08.460992    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:10.460493 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:10.459904    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:10.460493 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:10.460046    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:10.460493 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:10.460248    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:10.460493 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:10.460048    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:10.460493 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:10.460352    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:10.460493 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:10.460478    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:12.145264 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:12.145225    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:12.145713 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:12.145286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:12.145713 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.145467    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:12.145713 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.145531    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret podName:da4c2b44-97ef-47ec-812b-6f3bb1b5c536 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.145510749 +0000 UTC m=+18.114681343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret") pod "global-pull-secret-syncer-xpfh7" (UID: "da4c2b44-97ef-47ec-812b-6f3bb1b5c536") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:12.145980 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.145887    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:12.145980 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.145958    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs podName:79f55277-100d-44e8-b8e4-a1ca8c9ed7e5 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.145940712 +0000 UTC m=+18.115111301 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs") pod "network-metrics-daemon-bmllm" (UID: "79f55277-100d-44e8-b8e4-a1ca8c9ed7e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:12.246695 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:12.246660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:12.246877 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.246849    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:59:12.246877 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.246869    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:59:12.246877 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.246879    2563 projected.go:196] Error preparing data for projected volume kube-api-access-v7zwd for pod openshift-network-diagnostics/network-check-target-f7bgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:12.247038 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.246934    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd podName:6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.246918175 +0000 UTC m=+18.216088764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7zwd" (UniqueName: "kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd") pod "network-check-target-f7bgm" (UID: "6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:12.460349 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:12.460004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:12.460349 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.460114    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:12.460349 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:12.460182    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:12.460349 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.460303    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:12.460673 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:12.460355    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:12.460673 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:12.460435    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:14.459040 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:14.458751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:14.459529 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:14.459099    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:14.459529 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:14.459251    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:14.459529 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:14.459340    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:14.459529 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:14.459383    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:14.459784 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:14.459738    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:16.458991 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:16.458965    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:16.459419 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:16.458997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:16.459419 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:16.458965    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:16.459419 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:16.459066    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:16.459419 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:16.459147    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:16.459419 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:16.459242    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:18.459123 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:18.459085    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:18.459588 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:18.459085    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:18.459588 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:18.459221    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:18.459588 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:18.459095    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:18.459588 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:18.459305    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:18.459588 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:18.459378    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:20.207977 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:20.207940    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:20.208448 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:20.208028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:20.208448 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.208080    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:20.208448 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.208134    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:20.208448 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.208145    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret podName:da4c2b44-97ef-47ec-812b-6f3bb1b5c536 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:36.208128017 +0000 UTC m=+34.177298622 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret") pod "global-pull-secret-syncer-xpfh7" (UID: "da4c2b44-97ef-47ec-812b-6f3bb1b5c536") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:20.208448 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.208186    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs podName:79f55277-100d-44e8-b8e4-a1ca8c9ed7e5 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:36.208167722 +0000 UTC m=+34.177338304 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs") pod "network-metrics-daemon-bmllm" (UID: "79f55277-100d-44e8-b8e4-a1ca8c9ed7e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:20.308748 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:20.308725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:20.308848 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.308836    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:59:20.308890 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.308853    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:59:20.308890 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.308861    2563 projected.go:196] Error preparing data for projected volume kube-api-access-v7zwd for pod openshift-network-diagnostics/network-check-target-f7bgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:20.308954 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.308899    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd podName:6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:36.308886748 +0000 UTC m=+34.278057330 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7zwd" (UniqueName: "kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd") pod "network-check-target-f7bgm" (UID: "6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:20.459393 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:20.459305    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:20.459537 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:20.459309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:20.459537 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.459429    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:20.459537 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.459500    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:20.459537 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:20.459316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:20.459741 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:20.459568    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:22.459749 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:22.459714    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:22.460202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:22.459779    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:22.460202 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:22.459796    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:22.460202 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:22.459894    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:22.460202 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:22.459925    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:22.460202 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:22.460021    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:24.458701 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.458480    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:24.459211 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.458492    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:24.459211 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:24.458799    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:24.459211 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:24.458906    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:24.459211 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.458500    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:24.459211 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:24.459022    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:24.650407 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.650338    2563 generic.go:356] "Generic (PLEG): container finished" podID="b06f9c26-d0eb-4e4d-8424-51087b91fa18" containerID="2302ffdb376621a065eeadb729935b378b8cff50df8317ab7e3bdd15d1b81a58" exitCode=0
Sep 07 08:59:24.650407 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.650380    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdspn" event={"ID":"b06f9c26-d0eb-4e4d-8424-51087b91fa18","Type":"ContainerDied","Data":"2302ffdb376621a065eeadb729935b378b8cff50df8317ab7e3bdd15d1b81a58"}
Sep 07 08:59:24.651856 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.651831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m6gv9" event={"ID":"2767bb9a-644e-4a87-999c-a90f8916437b","Type":"ContainerStarted","Data":"17767a24e9c35ca4cdb934d9184ef2093be7c021ecd8d0ec92f350dbc1a530ae"}
Sep 07 08:59:24.654560 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.654541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" event={"ID":"762acdac-34ca-469a-87e9-e945fb096c34","Type":"ContainerStarted","Data":"e51c259df5b54509b54fb4b1b8a0ca7f1372b1b7abd4053cbcbec66a4a970cc0"}
Sep 07 08:59:24.654657 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.654599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" event={"ID":"762acdac-34ca-469a-87e9-e945fb096c34","Type":"ContainerStarted","Data":"5708ac114e1c04413a20532884b5d6bf031acda0e5b03e1a6df1752dcee8b357"}
Sep 07 08:59:24.654657 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.654630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" event={"ID":"762acdac-34ca-469a-87e9-e945fb096c34","Type":"ContainerStarted","Data":"1222229a21883ece452f8a995efc575e5fef2c30842cd0353a79afefc5d0c5ce"}
Sep 07 08:59:24.654767 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.654665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" event={"ID":"762acdac-34ca-469a-87e9-e945fb096c34","Type":"ContainerStarted","Data":"09526a2c9e2e6d57d67ab42c06a1ab820da0e833a6c6941a992103757d9893ee"}
Sep 07 08:59:24.654767 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.654694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" event={"ID":"762acdac-34ca-469a-87e9-e945fb096c34","Type":"ContainerStarted","Data":"92a70794fce6d84d9864e4b23ab2e913762166faf677eee2a0902d4bc27bd8a2"}
Sep 07 08:59:24.655999 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.655978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fkt8f" event={"ID":"83680078-ce21-45b5-b66c-c4211b91f06b","Type":"ContainerStarted","Data":"de7764dedb339bb182759776cf93235f653aca54121f7cca390cb1204af13a5f"}
Sep 07 08:59:24.657489 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.657469    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tg9lx" event={"ID":"53d15ca2-406b-450f-9fc2-196927235f32","Type":"ContainerStarted","Data":"2006c2c4f9b9b4bba46bddb52078e038c5f6181b7ce02198cd1b5b5f71763c04"}
Sep 07 08:59:24.658931 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.658911    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tbwx8" event={"ID":"f3397570-2652-435f-87ba-722b12bba06e","Type":"ContainerStarted","Data":"3e3cf6882d82338962c34c3a41deea3b96854b61dc95db9834449e7d781b171a"}
Sep 07 08:59:24.660325 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.660306    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm" event={"ID":"43e2c740-a35b-4b80-b15a-82e83b47e1db","Type":"ContainerStarted","Data":"00d8dd484ca579720f27861331fe6426bc40aed84d7d57ff1f2a95257d82f3b6"}
Sep 07 08:59:24.661731 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.661712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bw5zk" event={"ID":"2c8d36ac-5cab-446f-8e12-19f48927540c","Type":"ContainerStarted","Data":"7cf78fe107346c0d2e74649cdb5eca004ec2f638f80cd999c05f03bea743d507"}
Sep 07 08:59:24.701021 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.700984    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tg9lx" podStartSLOduration=3.515537375 podStartE2EDuration="21.700972042s" totalImagesPullingTime="18.185434667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:24.700672864 +0000 UTC m=+22.669843491" watchObservedRunningTime="2026-09-07 08:59:24.700972042 +0000 UTC m=+22.670142646"
Sep 07 08:59:24.701219 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.701200    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fkt8f" podStartSLOduration=3.534437575 podStartE2EDuration="21.701192953s" totalImagesPullingTime="18.166755378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:24.684567591 +0000 UTC m=+22.653738196" watchObservedRunningTime="2026-09-07 08:59:24.701192953 +0000 UTC m=+22.670363558"
Sep 07 08:59:24.717889 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.717852    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m6gv9" podStartSLOduration=6.8905414270000005 podStartE2EDuration="21.717839769s" totalImagesPullingTime="14.827298342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:24.717707082 +0000 UTC m=+22.686877686" watchObservedRunningTime="2026-09-07 08:59:24.717839769 +0000 UTC m=+22.687010401"
Sep 07 08:59:24.731296 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.731254    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bw5zk" podStartSLOduration=3.559269444 podStartE2EDuration="21.731244085s" totalImagesPullingTime="18.171974641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:24.731232322 +0000 UTC m=+22.700402927" watchObservedRunningTime="2026-09-07 08:59:24.731244085 +0000 UTC m=+22.700414671"
Sep 07 08:59:24.747393 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:24.747360    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tbwx8" podStartSLOduration=3.570324909 podStartE2EDuration="21.747349883s" totalImagesPullingTime="18.177024974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:24.747131369 +0000 UTC m=+22.716301974" watchObservedRunningTime="2026-09-07 08:59:24.747349883 +0000 UTC m=+22.716520505"
Sep 07 08:59:25.048144 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:25.047996    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 07 08:59:25.417701 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:25.417544    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T08:59:25.048013242Z","UUID":"d17dfb48-d155-4400-8448-a1f938faef6d","Handler":null,"Name":"","Endpoint":""}
Sep 07 08:59:25.422181 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:25.422048    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 07 08:59:25.422181 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:25.422089    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 07 08:59:25.667099 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:25.667062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" event={"ID":"762acdac-34ca-469a-87e9-e945fb096c34","Type":"ContainerStarted","Data":"0aec553564b9959043bd9269dc6958c43c280f9fcb7f412f94696fc5a1d57573"}
Sep 07 08:59:25.668854 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:25.668768    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kps6t" event={"ID":"4a9f8a1c-0d68-43ba-9d24-51290db60316","Type":"ContainerStarted","Data":"9f9360e7b605651664bc345ad7f9e00858f58f9d1806f0ca37ca5f4dddfd7084"}
Sep 07 08:59:25.671274 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:25.671247    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm" event={"ID":"43e2c740-a35b-4b80-b15a-82e83b47e1db","Type":"ContainerStarted","Data":"b10dc27b14f22aaa610d876ce3e6899f8aab597ca8c43f4763ff927bf7de8773"}
Sep 07 08:59:25.682801 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:25.682756    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kps6t" podStartSLOduration=4.510265455 podStartE2EDuration="22.682745365s" totalImagesPullingTime="18.17247991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:25.682740931 +0000 UTC m=+23.651911535" watchObservedRunningTime="2026-09-07 08:59:25.682745365 +0000 UTC m=+23.651915971"
Sep 07 08:59:26.459062 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:26.458970    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:26.459343 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:26.458969    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:26.459343 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:26.459071    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:26.459343 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:26.458982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:26.459343 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:26.459163    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:26.459343 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:26.459273    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:26.674759 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:26.674725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm" event={"ID":"43e2c740-a35b-4b80-b15a-82e83b47e1db","Type":"ContainerStarted","Data":"364ca56800ac22089c54770f0b7a7fa811219913251a01bca9b02c4f05d6b897"}
Sep 07 08:59:26.694583 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:26.694541    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vk2sm" podStartSLOduration=3.592340078 podStartE2EDuration="23.694522957s" totalImagesPullingTime="20.102182879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:26.693328552 +0000 UTC m=+24.662499157" watchObservedRunningTime="2026-09-07 08:59:26.694522957 +0000 UTC m=+24.663693561"
Sep 07 08:59:27.680874 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:27.680841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" event={"ID":"762acdac-34ca-469a-87e9-e945fb096c34","Type":"ContainerStarted","Data":"9bde0614d5bd34b525b6a8fa36b87af2ad804b7700380b4f5187d3c59179488f"}
Sep 07 08:59:28.093828 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:28.093752    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:28.094354 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:28.094331    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:28.256653 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:28.256621    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fkt8f_83680078-ce21-45b5-b66c-c4211b91f06b/dns-node-resolver/0.log"
Sep 07 08:59:28.458839 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:28.458771    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:28.458839 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:28.458813    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:28.459062 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:28.458776    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:28.459062 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:28.458925    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:28.459062 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:28.459027    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:28.459214 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:28.459086    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:28.682513 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:28.682484    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:28.683051 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:28.683029    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m6gv9"
Sep 07 08:59:29.440662 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:29.440463    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bw5zk_2c8d36ac-5cab-446f-8e12-19f48927540c/node-ca/0.log"
Sep 07 08:59:29.686369 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:29.686333    2563 generic.go:356] "Generic (PLEG): container finished" podID="b06f9c26-d0eb-4e4d-8424-51087b91fa18" containerID="1df49b7b21fbf61a1384608034b56c043b9da076553d53f79092e521c413bce4" exitCode=0
Sep 07 08:59:29.686900 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:29.686423    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdspn" event={"ID":"b06f9c26-d0eb-4e4d-8424-51087b91fa18","Type":"ContainerDied","Data":"1df49b7b21fbf61a1384608034b56c043b9da076553d53f79092e521c413bce4"}
Sep 07 08:59:29.690364 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:29.690333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" event={"ID":"762acdac-34ca-469a-87e9-e945fb096c34","Type":"ContainerStarted","Data":"f29a91c95619d71638f9b982b0e4a8cdb95bd724567ef3891330302143436e63"}
Sep 07 08:59:29.737151 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:29.737074    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs" podStartSLOduration=8.37730893 podStartE2EDuration="26.737060631s" totalImagesPullingTime="18.359751701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:29.735416981 +0000 UTC m=+27.704587586" watchObservedRunningTime="2026-09-07 08:59:29.737060631 +0000 UTC m=+27.706231244"
Sep 07 08:59:30.458722 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.458690    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:30.458874 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.458690    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:30.458874 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:30.458812    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:30.458988 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:30.458908    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:30.458988 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.458690    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:30.459075 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:30.459017    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:30.692477 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.692455    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 07 08:59:30.693171 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.693068    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:30.693325 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.693175    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:30.708172 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.708069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:30.708415 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.708398    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:30.898494 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.898462    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xpfh7"]
Sep 07 08:59:30.898652 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.898605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:30.898742 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:30.898718    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:30.899108 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.899087    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f7bgm"]
Sep 07 08:59:30.899211 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.899184    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:30.899307 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:30.899273    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:30.901086 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.901062    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bmllm"]
Sep 07 08:59:30.901179 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:30.901167    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:30.901324 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:30.901272    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:31.157050 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:31.157026    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 08:59:31.696490 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:31.696462    2563 generic.go:356] "Generic (PLEG): container finished" podID="b06f9c26-d0eb-4e4d-8424-51087b91fa18" containerID="3c483e26149fbbbaf4a619f4ebd179cb8b822786589742f42220469fdf90652c" exitCode=0
Sep 07 08:59:31.696932 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:31.696538    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdspn" event={"ID":"b06f9c26-d0eb-4e4d-8424-51087b91fa18","Type":"ContainerDied","Data":"3c483e26149fbbbaf4a619f4ebd179cb8b822786589742f42220469fdf90652c"}
Sep 07 08:59:32.459233 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:32.459202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:32.459473 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:32.459304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:32.459555 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:32.459476    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:32.459608 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:32.459562    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:32.459608 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:32.459352    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:32.459700 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:32.459662    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:33.703780 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:33.703748    2563 generic.go:356] "Generic (PLEG): container finished" podID="b06f9c26-d0eb-4e4d-8424-51087b91fa18" containerID="8df5082934102426f3fb944d38b765558d5f25283f428bd14cf543d222a5363d" exitCode=0
Sep 07 08:59:33.704151 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:33.703811    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdspn" event={"ID":"b06f9c26-d0eb-4e4d-8424-51087b91fa18","Type":"ContainerDied","Data":"8df5082934102426f3fb944d38b765558d5f25283f428bd14cf543d222a5363d"}
Sep 07 08:59:34.459338 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:34.459303    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:34.459338 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:34.459330    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:34.459568 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:34.459315    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:34.459568 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:34.459451    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:34.459665 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:34.459591    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:34.459728 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:34.459709    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:36.235727 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:36.235696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:36.236180 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:36.235738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:36.236180 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.235854    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:36.236180 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.235870    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:36.236180 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.235913    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs podName:79f55277-100d-44e8-b8e4-a1ca8c9ed7e5 nodeName:}" failed. No retries permitted until 2026-09-07 09:00:08.235897835 +0000 UTC m=+66.205068423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs") pod "network-metrics-daemon-bmllm" (UID: "79f55277-100d-44e8-b8e4-a1ca8c9ed7e5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:36.236180 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.235929    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret podName:da4c2b44-97ef-47ec-812b-6f3bb1b5c536 nodeName:}" failed. No retries permitted until 2026-09-07 09:00:08.235922644 +0000 UTC m=+66.205093230 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret") pod "global-pull-secret-syncer-xpfh7" (UID: "da4c2b44-97ef-47ec-812b-6f3bb1b5c536") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:36.336654 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:36.336626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:36.336866 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.336749    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:59:36.336866 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.336762    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:59:36.336866 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.336770    2563 projected.go:196] Error preparing data for projected volume kube-api-access-v7zwd for pod openshift-network-diagnostics/network-check-target-f7bgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:36.336866 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.336813    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd podName:6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2 nodeName:}" failed. No retries permitted until 2026-09-07 09:00:08.336801756 +0000 UTC m=+66.305972339 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7zwd" (UniqueName: "kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd") pod "network-check-target-f7bgm" (UID: "6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:36.459328 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:36.459269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:36.459328 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:36.459307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:36.459632 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:36.459273    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:36.459632 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.459387    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:36.459632 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.459473    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:36.459632 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:36.459536    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:38.458688 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:38.458544    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:38.458964 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:38.458544    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:38.458964 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:38.458759    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:38.458964 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:38.458576    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:38.458964 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:38.458858    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:38.458964 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:38.458921    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:38.717322 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:38.717233    2563 generic.go:356] "Generic (PLEG): container finished" podID="b06f9c26-d0eb-4e4d-8424-51087b91fa18" containerID="7485c063f869b96e7c55c8750ebde90c203beab8105ea6b2e3917e765ece6f20" exitCode=0
Sep 07 08:59:38.717322 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:38.717301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdspn" event={"ID":"b06f9c26-d0eb-4e4d-8424-51087b91fa18","Type":"ContainerDied","Data":"7485c063f869b96e7c55c8750ebde90c203beab8105ea6b2e3917e765ece6f20"}
Sep 07 08:59:39.722357 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:39.722321    2563 generic.go:356] "Generic (PLEG): container finished" podID="b06f9c26-d0eb-4e4d-8424-51087b91fa18" containerID="16efcea92aa40d0a7fa8bae5bea947431a29bf38fd6b7045d1cce4cf41048550" exitCode=0
Sep 07 08:59:39.722357 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:39.722361    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdspn" event={"ID":"b06f9c26-d0eb-4e4d-8424-51087b91fa18","Type":"ContainerDied","Data":"16efcea92aa40d0a7fa8bae5bea947431a29bf38fd6b7045d1cce4cf41048550"}
Sep 07 08:59:40.459148 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:40.459126    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:40.459278 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:40.459150    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:40.459278 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:40.459130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:40.459278 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:40.459221    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:40.459480 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:40.459338    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:40.459480 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:40.459387    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:40.728668 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:40.728596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fdspn" event={"ID":"b06f9c26-d0eb-4e4d-8424-51087b91fa18","Type":"ContainerStarted","Data":"911f6115877d484189457827dd3c0db9d23890aa84e3d8809a8e158c972cebf1"}
Sep 07 08:59:40.750067 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:40.750026    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fdspn" podStartSLOduration=8.969650909 podStartE2EDuration="37.750012034s" totalImagesPullingTime="28.780361125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:40.748444825 +0000 UTC m=+38.717615450" watchObservedRunningTime="2026-09-07 08:59:40.750012034 +0000 UTC m=+38.719182687"
Sep 07 08:59:42.459043 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:42.459012    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:42.459490 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:42.459082    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:42.459490 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:42.459091    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:42.459490 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:42.459202    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:42.459490 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:42.459251    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:42.459490 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:42.459356    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:44.458850 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:44.458811    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:44.458850 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:44.458839    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:44.459417 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:44.458811    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:44.459417 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:44.458935    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:44.459417 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:44.458983    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:44.459417 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:44.459064    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:46.459389 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:46.459353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:46.459873 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:46.459354    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:46.459873 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:46.459447    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:46.459873 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:46.459457    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:46.459873 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:46.459560    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:46.459873 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:46.459624    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:48.459194 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:48.459164    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:48.459194 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:48.459175    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:48.459194 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:48.459188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:48.459713 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:48.459274    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:48.459713 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:48.459368    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:48.459713 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:48.459496    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:50.458760 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:50.458724    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:50.458760 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:50.458749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:50.459161 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:50.458727    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:50.459161 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:50.458840    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f7bgm" podUID="6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2"
Sep 07 08:59:50.459161 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:50.458894    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xpfh7" podUID="da4c2b44-97ef-47ec-812b-6f3bb1b5c536"
Sep 07 08:59:50.459161 ip-10-0-138-157 kubenswrapper[2563]: E0907 08:59:50.458953    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmllm" podUID="79f55277-100d-44e8-b8e4-a1ca8c9ed7e5"
Sep 07 08:59:52.307472 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.307441    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-157.ec2.internal" event="NodeReady"
Sep 07 08:59:52.307849 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.307585    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 07 08:59:52.367426 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.367404    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d4c6c4978-zbzln"]
Sep 07 08:59:52.375294 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.375268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.377725 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.377703    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 07 08:59:52.377927 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.377911    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8sncs\""
Sep 07 08:59:52.377993 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.377943    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 07 08:59:52.377993 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.377981    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 07 08:59:52.382248 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.382229    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z7tq5"]
Sep 07 08:59:52.385731 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.385711    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 07 08:59:52.391530 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.391514    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n2qsg"]
Sep 07 08:59:52.391647 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.391629    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.394608 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.394379    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hmltn\""
Sep 07 08:59:52.394608 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.394467    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 07 08:59:52.395003 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.394844    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 07 08:59:52.401377 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.401360    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d4c6c4978-zbzln"]
Sep 07 08:59:52.401468 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.401385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z7tq5"]
Sep 07 08:59:52.401468 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.401403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n2qsg"]
Sep 07 08:59:52.401574 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.401484    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.404149 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.404129    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 07 08:59:52.404314 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.404212    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 07 08:59:52.404794 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.404776    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 07 08:59:52.405345 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.405327    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 07 08:59:52.405446 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.405392    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vhtvf\""
Sep 07 08:59:52.457126 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457093    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f50474e9-671c-414f-987d-50491a4978eb-data-volume\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.457126 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/809d141e-6429-49f6-b9a7-f3bcf0179714-trusted-ca\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.457302 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/809d141e-6429-49f6-b9a7-f3bcf0179714-installation-pull-secrets\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.457302 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/809d141e-6429-49f6-b9a7-f3bcf0179714-bound-sa-token\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.457302 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b4d281b-4835-48b0-b6ba-6f558f113bbc-tmp-dir\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.457302 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f50474e9-671c-414f-987d-50491a4978eb-crio-socket\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.457302 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457240    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f50474e9-671c-414f-987d-50491a4978eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.457498 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/809d141e-6429-49f6-b9a7-f3bcf0179714-image-registry-private-configuration\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.457498 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b4d281b-4835-48b0-b6ba-6f558f113bbc-config-volume\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.457498 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/809d141e-6429-49f6-b9a7-f3bcf0179714-registry-tls\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.457498 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/809d141e-6429-49f6-b9a7-f3bcf0179714-ca-trust-extracted\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.457498 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b52tw\" (UniqueName: \"kubernetes.io/projected/2b4d281b-4835-48b0-b6ba-6f558f113bbc-kube-api-access-b52tw\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.457498 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/809d141e-6429-49f6-b9a7-f3bcf0179714-registry-certificates\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.457498 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457482    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zlxv\" (UniqueName: \"kubernetes.io/projected/809d141e-6429-49f6-b9a7-f3bcf0179714-kube-api-access-6zlxv\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.457724 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b4d281b-4835-48b0-b6ba-6f558f113bbc-metrics-tls\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.457724 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f50474e9-671c-414f-987d-50491a4978eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.457724 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.457557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wz25\" (UniqueName: \"kubernetes.io/projected/f50474e9-671c-414f-987d-50491a4978eb-kube-api-access-6wz25\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.463815 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.463791    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 08:59:52.463935 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.463868    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 08:59:52.463935 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.463897    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 08:59:52.466640 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.466623    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlzfg\""
Sep 07 08:59:52.466843 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.466827    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 08:59:52.467001 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.466981    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 08:59:52.467351 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.467333    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 08:59:52.467456 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.467352    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 08:59:52.467558 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.467543    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-txgtt\""
Sep 07 08:59:52.476147 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.476130    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jvwmj"]
Sep 07 08:59:52.494839 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.494821    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jvwmj"]
Sep 07 08:59:52.494929 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.494894    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jvwmj"
Sep 07 08:59:52.497513 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.497495    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 07 08:59:52.497585 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.497557    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fc2n8\""
Sep 07 08:59:52.497674 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.497660    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 07 08:59:52.498005 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.497989    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 07 08:59:52.558467 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.558380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/809d141e-6429-49f6-b9a7-f3bcf0179714-registry-tls\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.558467 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.558451    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/809d141e-6429-49f6-b9a7-f3bcf0179714-ca-trust-extracted\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.558688 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.558478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b52tw\" (UniqueName: \"kubernetes.io/projected/2b4d281b-4835-48b0-b6ba-6f558f113bbc-kube-api-access-b52tw\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.558688 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.558499    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/809d141e-6429-49f6-b9a7-f3bcf0179714-registry-certificates\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.558688 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.558521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zlxv\" (UniqueName: \"kubernetes.io/projected/809d141e-6429-49f6-b9a7-f3bcf0179714-kube-api-access-6zlxv\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.558688 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.558554    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78fnc\" (UniqueName: \"kubernetes.io/projected/bb78a497-2227-45ef-bd86-fc1cdece6bae-kube-api-access-78fnc\") pod \"ingress-canary-jvwmj\" (UID: \"bb78a497-2227-45ef-bd86-fc1cdece6bae\") " pod="openshift-ingress-canary/ingress-canary-jvwmj"
Sep 07 08:59:52.558688 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.558626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b4d281b-4835-48b0-b6ba-6f558f113bbc-metrics-tls\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.558688 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.558653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f50474e9-671c-414f-987d-50491a4978eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.558978 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.558951    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/809d141e-6429-49f6-b9a7-f3bcf0179714-ca-trust-extracted\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.559131 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wz25\" (UniqueName: \"kubernetes.io/projected/f50474e9-671c-414f-987d-50491a4978eb-kube-api-access-6wz25\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.559311 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f50474e9-671c-414f-987d-50491a4978eb-data-volume\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.559311 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/809d141e-6429-49f6-b9a7-f3bcf0179714-trusted-ca\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.559311 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/809d141e-6429-49f6-b9a7-f3bcf0179714-installation-pull-secrets\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/809d141e-6429-49f6-b9a7-f3bcf0179714-bound-sa-token\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f50474e9-671c-414f-987d-50491a4978eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559350    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b4d281b-4835-48b0-b6ba-6f558f113bbc-tmp-dir\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559373    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f50474e9-671c-414f-987d-50491a4978eb-crio-socket\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f50474e9-671c-414f-987d-50491a4978eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559418    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/809d141e-6429-49f6-b9a7-f3bcf0179714-image-registry-private-configuration\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb78a497-2227-45ef-bd86-fc1cdece6bae-cert\") pod \"ingress-canary-jvwmj\" (UID: \"bb78a497-2227-45ef-bd86-fc1cdece6bae\") " pod="openshift-ingress-canary/ingress-canary-jvwmj"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/809d141e-6429-49f6-b9a7-f3bcf0179714-registry-certificates\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b4d281b-4835-48b0-b6ba-6f558f113bbc-config-volume\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.559547 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559526    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f50474e9-671c-414f-987d-50491a4978eb-data-volume\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.560259 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559606    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f50474e9-671c-414f-987d-50491a4978eb-crio-socket\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.560259 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.559876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b4d281b-4835-48b0-b6ba-6f558f113bbc-tmp-dir\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.560259 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.560005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b4d281b-4835-48b0-b6ba-6f558f113bbc-config-volume\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.560259 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.560170    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/809d141e-6429-49f6-b9a7-f3bcf0179714-trusted-ca\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.562583 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.562555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f50474e9-671c-414f-987d-50491a4978eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.562694 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.562607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b4d281b-4835-48b0-b6ba-6f558f113bbc-metrics-tls\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.562758 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.562705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/809d141e-6429-49f6-b9a7-f3bcf0179714-installation-pull-secrets\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.562758 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.562741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/809d141e-6429-49f6-b9a7-f3bcf0179714-image-registry-private-configuration\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.562847 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.562785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/809d141e-6429-49f6-b9a7-f3bcf0179714-registry-tls\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.566441 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.566420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b52tw\" (UniqueName: \"kubernetes.io/projected/2b4d281b-4835-48b0-b6ba-6f558f113bbc-kube-api-access-b52tw\") pod \"dns-default-z7tq5\" (UID: \"2b4d281b-4835-48b0-b6ba-6f558f113bbc\") " pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.566620 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.566604    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/809d141e-6429-49f6-b9a7-f3bcf0179714-bound-sa-token\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.566890 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.566874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zlxv\" (UniqueName: \"kubernetes.io/projected/809d141e-6429-49f6-b9a7-f3bcf0179714-kube-api-access-6zlxv\") pod \"image-registry-6d4c6c4978-zbzln\" (UID: \"809d141e-6429-49f6-b9a7-f3bcf0179714\") " pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.567212 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.567196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wz25\" (UniqueName: \"kubernetes.io/projected/f50474e9-671c-414f-987d-50491a4978eb-kube-api-access-6wz25\") pod \"insights-runtime-extractor-n2qsg\" (UID: \"f50474e9-671c-414f-987d-50491a4978eb\") " pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.660511 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.660472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78fnc\" (UniqueName: \"kubernetes.io/projected/bb78a497-2227-45ef-bd86-fc1cdece6bae-kube-api-access-78fnc\") pod \"ingress-canary-jvwmj\" (UID: \"bb78a497-2227-45ef-bd86-fc1cdece6bae\") " pod="openshift-ingress-canary/ingress-canary-jvwmj"
Sep 07 08:59:52.660650 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.660570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb78a497-2227-45ef-bd86-fc1cdece6bae-cert\") pod \"ingress-canary-jvwmj\" (UID: \"bb78a497-2227-45ef-bd86-fc1cdece6bae\") " pod="openshift-ingress-canary/ingress-canary-jvwmj"
Sep 07 08:59:52.662856 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.662834    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb78a497-2227-45ef-bd86-fc1cdece6bae-cert\") pod \"ingress-canary-jvwmj\" (UID: \"bb78a497-2227-45ef-bd86-fc1cdece6bae\") " pod="openshift-ingress-canary/ingress-canary-jvwmj"
Sep 07 08:59:52.668302 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.668263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78fnc\" (UniqueName: \"kubernetes.io/projected/bb78a497-2227-45ef-bd86-fc1cdece6bae-kube-api-access-78fnc\") pod \"ingress-canary-jvwmj\" (UID: \"bb78a497-2227-45ef-bd86-fc1cdece6bae\") " pod="openshift-ingress-canary/ingress-canary-jvwmj"
Sep 07 08:59:52.685117 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.685094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:52.700199 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.700175    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:52.709357 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.709320    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n2qsg"
Sep 07 08:59:52.802427 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:52.802397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jvwmj"
Sep 07 08:59:53.017883 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.017852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z7tq5"]
Sep 07 08:59:53.020892 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.020866    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d4c6c4978-zbzln"]
Sep 07 08:59:53.022339 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:53.022313    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b4d281b_4835_48b0_b6ba_6f558f113bbc.slice/crio-bfa45650af4c762a239c2b208090ea671ec3aa4c81e03bbefec534b015fbb842 WatchSource:0}: Error finding container bfa45650af4c762a239c2b208090ea671ec3aa4c81e03bbefec534b015fbb842: Status 404 returned error can't find the container with id bfa45650af4c762a239c2b208090ea671ec3aa4c81e03bbefec534b015fbb842
Sep 07 08:59:53.023235 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:53.023214    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809d141e_6429_49f6_b9a7_f3bcf0179714.slice/crio-de9b4e3a701269c527aee8e5e079c3be74c347d1d49f976a5b41487016631b77 WatchSource:0}: Error finding container de9b4e3a701269c527aee8e5e079c3be74c347d1d49f976a5b41487016631b77: Status 404 returned error can't find the container with id de9b4e3a701269c527aee8e5e079c3be74c347d1d49f976a5b41487016631b77
Sep 07 08:59:53.049688 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.049669    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n2qsg"]
Sep 07 08:59:53.057581 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:53.057559    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf50474e9_671c_414f_987d_50491a4978eb.slice/crio-0b3a53d8f75a3f3ca4a72d61e4a43d52a6e32a975d745da7784c4653f4fc0e23 WatchSource:0}: Error finding container 0b3a53d8f75a3f3ca4a72d61e4a43d52a6e32a975d745da7784c4653f4fc0e23: Status 404 returned error can't find the container with id 0b3a53d8f75a3f3ca4a72d61e4a43d52a6e32a975d745da7784c4653f4fc0e23
Sep 07 08:59:53.095588 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.095568    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jvwmj"]
Sep 07 08:59:53.098727 ip-10-0-138-157 kubenswrapper[2563]: W0907 08:59:53.098705    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb78a497_2227_45ef_bd86_fc1cdece6bae.slice/crio-3c2e1bbaa0239d2ae1c79045b5a91f039c188007ed7ab09242967a8e6b201e12 WatchSource:0}: Error finding container 3c2e1bbaa0239d2ae1c79045b5a91f039c188007ed7ab09242967a8e6b201e12: Status 404 returned error can't find the container with id 3c2e1bbaa0239d2ae1c79045b5a91f039c188007ed7ab09242967a8e6b201e12
Sep 07 08:59:53.759558 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.759498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2qsg" event={"ID":"f50474e9-671c-414f-987d-50491a4978eb","Type":"ContainerStarted","Data":"80368b4bd99b59b5ec2dc6d61129766d89a523bc6e67d66550c395eed3aeeabc"}
Sep 07 08:59:53.760026 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.759569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2qsg" event={"ID":"f50474e9-671c-414f-987d-50491a4978eb","Type":"ContainerStarted","Data":"0b3a53d8f75a3f3ca4a72d61e4a43d52a6e32a975d745da7784c4653f4fc0e23"}
Sep 07 08:59:53.761271 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.761236    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z7tq5" event={"ID":"2b4d281b-4835-48b0-b6ba-6f558f113bbc","Type":"ContainerStarted","Data":"bfa45650af4c762a239c2b208090ea671ec3aa4c81e03bbefec534b015fbb842"}
Sep 07 08:59:53.763544 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.763520    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln" event={"ID":"809d141e-6429-49f6-b9a7-f3bcf0179714","Type":"ContainerStarted","Data":"98ec79d37cc3121f31fbf14b07076d2a40dbcccac3b38f14ce8f990a325bcaf3"}
Sep 07 08:59:53.763649 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.763565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln" event={"ID":"809d141e-6429-49f6-b9a7-f3bcf0179714","Type":"ContainerStarted","Data":"de9b4e3a701269c527aee8e5e079c3be74c347d1d49f976a5b41487016631b77"}
Sep 07 08:59:53.763649 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.763614    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 08:59:53.765259 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.765235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jvwmj" event={"ID":"bb78a497-2227-45ef-bd86-fc1cdece6bae","Type":"ContainerStarted","Data":"3c2e1bbaa0239d2ae1c79045b5a91f039c188007ed7ab09242967a8e6b201e12"}
Sep 07 08:59:53.783714 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:53.783662    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln" podStartSLOduration=1.78364309 podStartE2EDuration="1.78364309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:53.78295978 +0000 UTC m=+51.752130386" watchObservedRunningTime="2026-09-07 08:59:53.78364309 +0000 UTC m=+51.752813696"
Sep 07 08:59:54.770638 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:54.770541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2qsg" event={"ID":"f50474e9-671c-414f-987d-50491a4978eb","Type":"ContainerStarted","Data":"631a292117f66e33e121ce8283feaf05ce507420cf1b7932eaea6e432dd194ca"}
Sep 07 08:59:55.774783 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:55.774740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z7tq5" event={"ID":"2b4d281b-4835-48b0-b6ba-6f558f113bbc","Type":"ContainerStarted","Data":"243f6b1d1bfed161d6319b9c6117c350742ea03b2f2d9c4158faea6515b9c1b1"}
Sep 07 08:59:56.779468 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:56.779428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jvwmj" event={"ID":"bb78a497-2227-45ef-bd86-fc1cdece6bae","Type":"ContainerStarted","Data":"5cc2194cee2918191ec2ecf73d9507c1a53a2b31c8619acd1a5656495b4d4869"}
Sep 07 08:59:56.781428 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:56.781369    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2qsg" event={"ID":"f50474e9-671c-414f-987d-50491a4978eb","Type":"ContainerStarted","Data":"0d27f55b70232252f2947704a3324c7b9c6cf2a25c9e1c02cd4619865342495b"}
Sep 07 08:59:56.782925 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:56.782903    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z7tq5" event={"ID":"2b4d281b-4835-48b0-b6ba-6f558f113bbc","Type":"ContainerStarted","Data":"97bd2ea29108a33b63bb4c12fbb77dd72d2361ed6f95eed5f469e59ee0a23368"}
Sep 07 08:59:56.783058 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:56.783042    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z7tq5"
Sep 07 08:59:56.794565 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:56.794521    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jvwmj" podStartSLOduration=1.945932081 podStartE2EDuration="4.79450958s" totalImagesPullingTime="2.848577499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:56.793699332 +0000 UTC m=+54.762869936" watchObservedRunningTime="2026-09-07 08:59:56.79450958 +0000 UTC m=+54.763680210"
Sep 07 08:59:56.814433 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:56.811615    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n2qsg" podStartSLOduration=2.097978738 podStartE2EDuration="4.811597463s" totalImagesPullingTime="2.713618725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:56.81024164 +0000 UTC m=+54.779412260" watchObservedRunningTime="2026-09-07 08:59:56.811597463 +0000 UTC m=+54.780768069"
Sep 07 08:59:56.828575 ip-10-0-138-157 kubenswrapper[2563]: I0907 08:59:56.828531    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z7tq5" podStartSLOduration=3.1684316949999998 podStartE2EDuration="4.82851547s" totalImagesPullingTime="1.660083775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:56.827987218 +0000 UTC m=+54.797157824" watchObservedRunningTime="2026-09-07 08:59:56.82851547 +0000 UTC m=+54.797686074"
Sep 07 09:00:02.709350 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:02.709323    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vvtcs"
Sep 07 09:00:03.090712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.090676    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9xjcg"]
Sep 07 09:00:03.094038 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.094021    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.101620 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.101589    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 07 09:00:03.101741 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.101649    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 07 09:00:03.101741 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.101650    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 07 09:00:03.101741 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.101725    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 07 09:00:03.101997 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.101979    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 07 09:00:03.102096 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.102081    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jcq2k\""
Sep 07 09:00:03.102137 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.102127    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 07 09:00:03.132272 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.132243    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-wtmp\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.132416 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.132304    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.132416 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.132386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/452b70d9-3fa4-49a4-a65b-5688bdef59c0-root\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.132510 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.132417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/452b70d9-3fa4-49a4-a65b-5688bdef59c0-sys\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.132510 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.132440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-textfile\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.132510 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.132455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/452b70d9-3fa4-49a4-a65b-5688bdef59c0-metrics-client-ca\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.132603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.132508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.132603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.132535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-tls\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.132603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.132553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz28h\" (UniqueName: \"kubernetes.io/projected/452b70d9-3fa4-49a4-a65b-5688bdef59c0-kube-api-access-dz28h\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233620 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233785 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/452b70d9-3fa4-49a4-a65b-5688bdef59c0-root\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233785 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/452b70d9-3fa4-49a4-a65b-5688bdef59c0-sys\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233785 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-textfile\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233785 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/452b70d9-3fa4-49a4-a65b-5688bdef59c0-metrics-client-ca\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/452b70d9-3fa4-49a4-a65b-5688bdef59c0-root\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233794    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/452b70d9-3fa4-49a4-a65b-5688bdef59c0-sys\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-tls\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.233991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.233969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz28h\" (UniqueName: \"kubernetes.io/projected/452b70d9-3fa4-49a4-a65b-5688bdef59c0-kube-api-access-dz28h\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.234210 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.234015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-wtmp\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.234210 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:00:03.234106    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 07 09:00:03.234210 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.234153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-textfile\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.234210 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.234164    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-wtmp\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.234210 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:00:03.234169    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-tls podName:452b70d9-3fa4-49a4-a65b-5688bdef59c0 nodeName:}" failed. No retries permitted until 2026-09-07 09:00:03.73414877 +0000 UTC m=+61.703319377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-tls") pod "node-exporter-9xjcg" (UID: "452b70d9-3fa4-49a4-a65b-5688bdef59c0") : secret "node-exporter-tls" not found
Sep 07 09:00:03.234462 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.234442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/452b70d9-3fa4-49a4-a65b-5688bdef59c0-metrics-client-ca\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.234500 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.234446    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.236039 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.236016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.245990 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.245965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz28h\" (UniqueName: \"kubernetes.io/projected/452b70d9-3fa4-49a4-a65b-5688bdef59c0-kube-api-access-dz28h\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.737583 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.737536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-tls\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:03.739863 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:03.739843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/452b70d9-3fa4-49a4-a65b-5688bdef59c0-node-exporter-tls\") pod \"node-exporter-9xjcg\" (UID: \"452b70d9-3fa4-49a4-a65b-5688bdef59c0\") " pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:04.003114 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.003082    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9xjcg"
Sep 07 09:00:04.013548 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:00:04.013519    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452b70d9_3fa4_49a4_a65b_5688bdef59c0.slice/crio-802fda1141dab77d663ef993f708c7e0d3f50763051bf41b8173b34cc3b7b0b7 WatchSource:0}: Error finding container 802fda1141dab77d663ef993f708c7e0d3f50763051bf41b8173b34cc3b7b0b7: Status 404 returned error can't find the container with id 802fda1141dab77d663ef993f708c7e0d3f50763051bf41b8173b34cc3b7b0b7
Sep 07 09:00:04.141544 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.141490    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 09:00:04.144278 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.144260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.146681 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.146658    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 07 09:00:04.147060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.147040    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 07 09:00:04.147223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.147087    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 07 09:00:04.147382 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.147273    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 07 09:00:04.147382 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.147350    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 07 09:00:04.147382 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.147347    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 07 09:00:04.147382 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.147351    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 07 09:00:04.147653 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.147368    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 07 09:00:04.147714 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.147665    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 07 09:00:04.147714 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.147691    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fddj5\""
Sep 07 09:00:04.163005 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.162976    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 09:00:04.240906 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.240870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.240906 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.240907    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvthq\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-kube-api-access-hvthq\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.240927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-web-config\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.240998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.241033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.241095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-volume\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241252 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.241122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241252 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.241141    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241252 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.241167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-out\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241252 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.241185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241252 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.241238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241444 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.241259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.241444 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.241298    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342444 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342363    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-out\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342444 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342656 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:00:04.342608    2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 07 09:00:04.342890 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:00:04.342734    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-main-tls podName:f588f39c-0d9d-4dd3-8ef9-b46a43873260 nodeName:}" failed. No retries permitted until 2026-09-07 09:00:04.842676792 +0000 UTC m=+62.811847390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260") : secret "alertmanager-main-tls" not found
Sep 07 09:00:04.342890 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvthq\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-kube-api-access-hvthq\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342890 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-web-config\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342890 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342890 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342862    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.342890 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342872    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.343206 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-volume\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.343206 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.343206 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.342974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.344110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.344080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.344596 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.344568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.345551 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.345524    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-out\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.345683 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.345657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.345802 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.345667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-volume\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.346138 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.346113    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.346216 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.346147    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.346216 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.346177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.346216 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.346203    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-web-config\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.347102 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.347085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.350633 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.350613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvthq\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-kube-api-access-hvthq\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.810210 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.810172    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9xjcg" event={"ID":"452b70d9-3fa4-49a4-a65b-5688bdef59c0","Type":"ContainerStarted","Data":"802fda1141dab77d663ef993f708c7e0d3f50763051bf41b8173b34cc3b7b0b7"}
Sep 07 09:00:04.848384 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.848351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:04.850606 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:04.850581    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:05.053910 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.053879    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:00:05.147958 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.147926    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"]
Sep 07 09:00:05.150916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.150891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.153733 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.153711    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 07 09:00:05.154265 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.154245    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8s6cw\""
Sep 07 09:00:05.154393 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.154323    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 07 09:00:05.154449 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.154406    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e20jbuf223iho\""
Sep 07 09:00:05.154527 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.154511    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 07 09:00:05.154598 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.154579    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 07 09:00:05.154598 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.154598    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 07 09:00:05.165211 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.165179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"]
Sep 07 09:00:05.251537 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.251489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.251693 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.251543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46vr2\" (UniqueName: \"kubernetes.io/projected/bc14e598-469a-4994-b597-583395e97815-kube-api-access-46vr2\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.251693 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.251581    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.251693 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.251660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.251869 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.251714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-grpc-tls\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.251869 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.251813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-tls\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.251973 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.251880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.251973 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.251908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc14e598-469a-4994-b597-583395e97815-metrics-client-ca\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.336332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.336212    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 09:00:05.340254 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:00:05.340226    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf588f39c_0d9d_4dd3_8ef9_b46a43873260.slice/crio-dd75d908f3953a140f7aa1fe5b5dd54293f0b6a1edf609063f4e840cef751d0e WatchSource:0}: Error finding container dd75d908f3953a140f7aa1fe5b5dd54293f0b6a1edf609063f4e840cef751d0e: Status 404 returned error can't find the container with id dd75d908f3953a140f7aa1fe5b5dd54293f0b6a1edf609063f4e840cef751d0e
Sep 07 09:00:05.352378 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.352355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.352465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.352388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46vr2\" (UniqueName: \"kubernetes.io/projected/bc14e598-469a-4994-b597-583395e97815-kube-api-access-46vr2\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.352465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.352420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.352465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.352457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.352603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.352473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-grpc-tls\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.352603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.352517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-tls\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.352603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.352565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.352603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.352589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc14e598-469a-4994-b597-583395e97815-metrics-client-ca\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.353401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.353375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc14e598-469a-4994-b597-583395e97815-metrics-client-ca\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.355057 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.355026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.355150 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.355054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.355212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.355200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.355421 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.355399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.355483 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.355449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-thanos-querier-tls\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.355523 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.355483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bc14e598-469a-4994-b597-583395e97815-secret-grpc-tls\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.363000 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.362970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46vr2\" (UniqueName: \"kubernetes.io/projected/bc14e598-469a-4994-b597-583395e97815-kube-api-access-46vr2\") pod \"thanos-querier-789cbcdfc6-drpk5\" (UID: \"bc14e598-469a-4994-b597-583395e97815\") " pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.461913 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.461890    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:05.733184 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.733157    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"]
Sep 07 09:00:05.735448 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:00:05.735424    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc14e598_469a_4994_b597_583395e97815.slice/crio-2fa29ea8cbebc32430439c3ba0e739921c9e513787e3cd17c4ae2ccb643d2b84 WatchSource:0}: Error finding container 2fa29ea8cbebc32430439c3ba0e739921c9e513787e3cd17c4ae2ccb643d2b84: Status 404 returned error can't find the container with id 2fa29ea8cbebc32430439c3ba0e739921c9e513787e3cd17c4ae2ccb643d2b84
Sep 07 09:00:05.815872 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.815840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerStarted","Data":"dd75d908f3953a140f7aa1fe5b5dd54293f0b6a1edf609063f4e840cef751d0e"}
Sep 07 09:00:05.817342 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.817319    2563 generic.go:356] "Generic (PLEG): container finished" podID="452b70d9-3fa4-49a4-a65b-5688bdef59c0" containerID="b04d44b36919b37791b43d4221f08b5b24e443e0f8386c41707f556b9299bfed" exitCode=0
Sep 07 09:00:05.817443 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.817400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9xjcg" event={"ID":"452b70d9-3fa4-49a4-a65b-5688bdef59c0","Type":"ContainerDied","Data":"b04d44b36919b37791b43d4221f08b5b24e443e0f8386c41707f556b9299bfed"}
Sep 07 09:00:05.818439 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:05.818420    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5" event={"ID":"bc14e598-469a-4994-b597-583395e97815","Type":"ContainerStarted","Data":"2fa29ea8cbebc32430439c3ba0e739921c9e513787e3cd17c4ae2ccb643d2b84"}
Sep 07 09:00:06.788970 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:06.788943    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z7tq5"
Sep 07 09:00:06.827663 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:06.827626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9xjcg" event={"ID":"452b70d9-3fa4-49a4-a65b-5688bdef59c0","Type":"ContainerStarted","Data":"2170ba9b7c63f7099a5c6952e27dfc750e2f910b524eff1763c4b373c3764630"}
Sep 07 09:00:06.828064 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:06.827688    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9xjcg" event={"ID":"452b70d9-3fa4-49a4-a65b-5688bdef59c0","Type":"ContainerStarted","Data":"faf456626e91f31bc8abd70fa069bd59e99416a15f4516c5218010496206a8e9"}
Sep 07 09:00:06.847610 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:06.847552    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9xjcg" podStartSLOduration=2.488899168 podStartE2EDuration="3.847532915s" totalImagesPullingTime="1.298034175s" totalInitContainerRuntime="60.599572ms" isStatefulPod=true podCreationTimestamp="2026-09-07 09:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:06.846186354 +0000 UTC m=+64.815356988" watchObservedRunningTime="2026-09-07 09:00:06.847532915 +0000 UTC m=+64.816703518"
Sep 07 09:00:07.467142 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.467073    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79b8f874c5-5dq4f"]
Sep 07 09:00:07.470065 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.470034    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.472696 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.472668    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 07 09:00:07.472832 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.472670    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 07 09:00:07.473923 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.473739    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 07 09:00:07.473923 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.473752    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bj3257p5107m2\""
Sep 07 09:00:07.473923 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.473821    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 07 09:00:07.474183 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.474164    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-57wgb\""
Sep 07 09:00:07.481081 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.481057    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79b8f874c5-5dq4f"]
Sep 07 09:00:07.569885 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.569859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/73de73c6-b550-41e0-a5d4-8e921102b155-secret-metrics-server-client-certs\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.570021 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.569917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/73de73c6-b550-41e0-a5d4-8e921102b155-audit-log\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.570021 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.569978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/73de73c6-b550-41e0-a5d4-8e921102b155-metrics-server-audit-profiles\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.570130 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.570023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73de73c6-b550-41e0-a5d4-8e921102b155-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.570130 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.570043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73de73c6-b550-41e0-a5d4-8e921102b155-client-ca-bundle\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.570130 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.570114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/73de73c6-b550-41e0-a5d4-8e921102b155-secret-metrics-server-tls\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.570276 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.570210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncsg5\" (UniqueName: \"kubernetes.io/projected/73de73c6-b550-41e0-a5d4-8e921102b155-kube-api-access-ncsg5\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.670771 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.670736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncsg5\" (UniqueName: \"kubernetes.io/projected/73de73c6-b550-41e0-a5d4-8e921102b155-kube-api-access-ncsg5\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.670771 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.670775    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/73de73c6-b550-41e0-a5d4-8e921102b155-secret-metrics-server-client-certs\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.671006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.670891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/73de73c6-b550-41e0-a5d4-8e921102b155-audit-log\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.671006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.670914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/73de73c6-b550-41e0-a5d4-8e921102b155-metrics-server-audit-profiles\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.671006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.670938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73de73c6-b550-41e0-a5d4-8e921102b155-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.671006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.670959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73de73c6-b550-41e0-a5d4-8e921102b155-client-ca-bundle\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.671006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.671005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/73de73c6-b550-41e0-a5d4-8e921102b155-secret-metrics-server-tls\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.671342 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.671318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/73de73c6-b550-41e0-a5d4-8e921102b155-audit-log\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.671826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.671796    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73de73c6-b550-41e0-a5d4-8e921102b155-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.671983 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.671959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/73de73c6-b550-41e0-a5d4-8e921102b155-metrics-server-audit-profiles\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.673423 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.673397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/73de73c6-b550-41e0-a5d4-8e921102b155-secret-metrics-server-client-certs\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.673522 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.673437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/73de73c6-b550-41e0-a5d4-8e921102b155-secret-metrics-server-tls\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.673522 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.673509    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73de73c6-b550-41e0-a5d4-8e921102b155-client-ca-bundle\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.678437 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.678413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncsg5\" (UniqueName: \"kubernetes.io/projected/73de73c6-b550-41e0-a5d4-8e921102b155-kube-api-access-ncsg5\") pod \"metrics-server-79b8f874c5-5dq4f\" (UID: \"73de73c6-b550-41e0-a5d4-8e921102b155\") " pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.780658 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.780634    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:07.833779 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.833745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5" event={"ID":"bc14e598-469a-4994-b597-583395e97815","Type":"ContainerStarted","Data":"cd72db11544986be5d04949542170514f8d5f389bc796607e486a9a2534a70a2"}
Sep 07 09:00:07.834075 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.833838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5" event={"ID":"bc14e598-469a-4994-b597-583395e97815","Type":"ContainerStarted","Data":"4d7c18fde7b76b8fcf11eaf7755541b5018125232c6c3dd2f234700eb1c5f9a2"}
Sep 07 09:00:07.835991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.835964    2563 generic.go:356] "Generic (PLEG): container finished" podID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerID="d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2" exitCode=0
Sep 07 09:00:07.836188 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.836111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerDied","Data":"d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2"}
Sep 07 09:00:07.845808 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.844631    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"]
Sep 07 09:00:07.848090 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.848071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:07.851165 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.851069    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8ndx6\""
Sep 07 09:00:07.851165 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.851120    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"]
Sep 07 09:00:07.851369 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.851180    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 07 09:00:07.973800 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.973731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c3b71592-f725-4636-8ada-7e6a51564fcc-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-rq2rh\" (UID: \"c3b71592-f725-4636-8ada-7e6a51564fcc\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:07.973912 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:07.973837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96h94\" (UniqueName: \"kubernetes.io/projected/c3b71592-f725-4636-8ada-7e6a51564fcc-kube-api-access-96h94\") pod \"monitoring-plugin-85bbbbbf4-rq2rh\" (UID: \"c3b71592-f725-4636-8ada-7e6a51564fcc\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:08.061806 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.061778    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79b8f874c5-5dq4f"]
Sep 07 09:00:08.065066 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:00:08.065037    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73de73c6_b550_41e0_a5d4_8e921102b155.slice/crio-3ef3f78f53dfc8944b88416d6ea817ca81373331618606f72c9517aa4205a044 WatchSource:0}: Error finding container 3ef3f78f53dfc8944b88416d6ea817ca81373331618606f72c9517aa4205a044: Status 404 returned error can't find the container with id 3ef3f78f53dfc8944b88416d6ea817ca81373331618606f72c9517aa4205a044
Sep 07 09:00:08.075175 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.075147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c3b71592-f725-4636-8ada-7e6a51564fcc-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-rq2rh\" (UID: \"c3b71592-f725-4636-8ada-7e6a51564fcc\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:08.075320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.075271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96h94\" (UniqueName: \"kubernetes.io/projected/c3b71592-f725-4636-8ada-7e6a51564fcc-kube-api-access-96h94\") pod \"monitoring-plugin-85bbbbbf4-rq2rh\" (UID: \"c3b71592-f725-4636-8ada-7e6a51564fcc\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:08.077644 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.077619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c3b71592-f725-4636-8ada-7e6a51564fcc-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-rq2rh\" (UID: \"c3b71592-f725-4636-8ada-7e6a51564fcc\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:08.084755 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.084732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96h94\" (UniqueName: \"kubernetes.io/projected/c3b71592-f725-4636-8ada-7e6a51564fcc-kube-api-access-96h94\") pod \"monitoring-plugin-85bbbbbf4-rq2rh\" (UID: \"c3b71592-f725-4636-8ada-7e6a51564fcc\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:08.157968 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.157944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:08.277855 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.277823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 09:00:08.277972 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.277861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 09:00:08.280375 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.280358    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 09:00:08.280439 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.280424    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 09:00:08.290479 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.290453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da4c2b44-97ef-47ec-812b-6f3bb1b5c536-original-pull-secret\") pod \"global-pull-secret-syncer-xpfh7\" (UID: \"da4c2b44-97ef-47ec-812b-6f3bb1b5c536\") " pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 09:00:08.290621 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.290604    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f55277-100d-44e8-b8e4-a1ca8c9ed7e5-metrics-certs\") pod \"network-metrics-daemon-bmllm\" (UID: \"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5\") " pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 09:00:08.375539 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.375520    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xpfh7"
Sep 07 09:00:08.378636 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.378614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 09:00:08.381633 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.381616    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 09:00:08.386465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.386446    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-txgtt\""
Sep 07 09:00:08.392585 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.392556    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 09:00:08.394680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.394650    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmllm"
Sep 07 09:00:08.403684 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.403661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7zwd\" (UniqueName: \"kubernetes.io/projected/6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2-kube-api-access-v7zwd\") pod \"network-check-target-f7bgm\" (UID: \"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2\") " pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 09:00:08.438257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.438183    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"]
Sep 07 09:00:08.443628 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:00:08.443581    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b71592_f725_4636_8ada_7e6a51564fcc.slice/crio-426a0aa11c9102e57cbf7ae47f8da330cc8b86c8ade57c9dbb8a79589d01d18e WatchSource:0}: Error finding container 426a0aa11c9102e57cbf7ae47f8da330cc8b86c8ade57c9dbb8a79589d01d18e: Status 404 returned error can't find the container with id 426a0aa11c9102e57cbf7ae47f8da330cc8b86c8ade57c9dbb8a79589d01d18e
Sep 07 09:00:08.651156 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.651120    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xpfh7"]
Sep 07 09:00:08.680745 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.680718    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bmllm"]
Sep 07 09:00:08.691934 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.691904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlzfg\""
Sep 07 09:00:08.699902 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.699877    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 09:00:08.708141 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:00:08.708111    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda4c2b44_97ef_47ec_812b_6f3bb1b5c536.slice/crio-9963f51987e8e63104fe444c8c29a38b55fd0aa778989c11d8e47d410b646381 WatchSource:0}: Error finding container 9963f51987e8e63104fe444c8c29a38b55fd0aa778989c11d8e47d410b646381: Status 404 returned error can't find the container with id 9963f51987e8e63104fe444c8c29a38b55fd0aa778989c11d8e47d410b646381
Sep 07 09:00:08.708873 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:00:08.708838    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f55277_100d_44e8_b8e4_a1ca8c9ed7e5.slice/crio-398ffbf170a40c50fa528d2dca505fd83cb3b79e20265899cfa8ba7759c8d6bb WatchSource:0}: Error finding container 398ffbf170a40c50fa528d2dca505fd83cb3b79e20265899cfa8ba7759c8d6bb: Status 404 returned error can't find the container with id 398ffbf170a40c50fa528d2dca505fd83cb3b79e20265899cfa8ba7759c8d6bb
Sep 07 09:00:08.841462 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.841406    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f" event={"ID":"73de73c6-b550-41e0-a5d4-8e921102b155","Type":"ContainerStarted","Data":"3ef3f78f53dfc8944b88416d6ea817ca81373331618606f72c9517aa4205a044"}
Sep 07 09:00:08.844622 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.844583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5" event={"ID":"bc14e598-469a-4994-b597-583395e97815","Type":"ContainerStarted","Data":"456ab7c37f698a12b5d467676ab11c334cb68e428cdd6cb8f7c6899210f46926"}
Sep 07 09:00:08.844722 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.844656    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5" event={"ID":"bc14e598-469a-4994-b597-583395e97815","Type":"ContainerStarted","Data":"3831451dca15cb116f76445519b4fbc8856ffce5acc5bb7f7b12034fab9205de"}
Sep 07 09:00:08.846430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.846400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bmllm" event={"ID":"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5","Type":"ContainerStarted","Data":"398ffbf170a40c50fa528d2dca505fd83cb3b79e20265899cfa8ba7759c8d6bb"}
Sep 07 09:00:08.848762 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.848732    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xpfh7" event={"ID":"da4c2b44-97ef-47ec-812b-6f3bb1b5c536","Type":"ContainerStarted","Data":"9963f51987e8e63104fe444c8c29a38b55fd0aa778989c11d8e47d410b646381"}
Sep 07 09:00:08.850257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.850236    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh" event={"ID":"c3b71592-f725-4636-8ada-7e6a51564fcc","Type":"ContainerStarted","Data":"426a0aa11c9102e57cbf7ae47f8da330cc8b86c8ade57c9dbb8a79589d01d18e"}
Sep 07 09:00:08.984161 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:08.984114    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f7bgm"]
Sep 07 09:00:09.225590 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:00:09.225548    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c610c6c_2b93_4fcd_b3aa_9d971fe6c9a2.slice/crio-56eda0fff55c99f5a556895ddc6049ccc45ae0f6d510bb9d0590c880b792c4e7 WatchSource:0}: Error finding container 56eda0fff55c99f5a556895ddc6049ccc45ae0f6d510bb9d0590c880b792c4e7: Status 404 returned error can't find the container with id 56eda0fff55c99f5a556895ddc6049ccc45ae0f6d510bb9d0590c880b792c4e7
Sep 07 09:00:09.431459 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.431417    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 09:00:09.438176 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.437315    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.440644 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.440521    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 07 09:00:09.440857 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.440836    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 07 09:00:09.441114 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.440958    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1rchugldo3igm\""
Sep 07 09:00:09.442081 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.442054    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 07 09:00:09.442198 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.442185    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 07 09:00:09.442471 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.442452    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 07 09:00:09.442702 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.442682    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 07 09:00:09.442766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.442711    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 07 09:00:09.446256 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.446240    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 07 09:00:09.446472 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.446389    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-85wrm\""
Sep 07 09:00:09.446686 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.446668    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 07 09:00:09.447730 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.446860    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 07 09:00:09.447730 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.447032    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 07 09:00:09.447730 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.447328    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 07 09:00:09.447730 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.447597    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 07 09:00:09.451300 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.451263    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 07 09:00:09.452891 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.452870    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 09:00:09.489259 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489259 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489436 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489304    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489436 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-web-config\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489436 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489436 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489629 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489629 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489629 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489629 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config-out\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489629 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489629 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489642    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wtp6\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-kube-api-access-9wtp6\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.489920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.489865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590473 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-web-config\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590627    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config-out\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.590922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.591307 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.591307 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.591307 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.590979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.591307 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.591004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wtp6\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-kube-api-access-9wtp6\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.591848 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.591816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.593208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.593021    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.596759 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.593785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.596759 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.594616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.596759 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.594852    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.596759 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.595067    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.596759 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.595097    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.596759 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.595666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.596759 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.596622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.597779 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.597730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.598425 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.598373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.598519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.598429    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.598590 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.598553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config-out\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.598884 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.598860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.599110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.598863    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.599110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.598937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.599441 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.599419    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.600267 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.600035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.600267 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.600238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-web-config\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.612916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.612894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wtp6\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-kube-api-access-9wtp6\") pod \"prometheus-k8s-0\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.751716 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.751682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:09.858364 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.858323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5" event={"ID":"bc14e598-469a-4994-b597-583395e97815","Type":"ContainerStarted","Data":"45de7fc6e4347b726e4a3ebe5663305c55388ac9eec7bbb41892e67396af2d7f"}
Sep 07 09:00:09.858760 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.858415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5" event={"ID":"bc14e598-469a-4994-b597-583395e97815","Type":"ContainerStarted","Data":"373602a6a080fe85b7f6d32c8e9cb04bde504b7b5f88b97f79f4ccc5ff8db412"}
Sep 07 09:00:09.858760 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.858617    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:09.859737 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.859713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f7bgm" event={"ID":"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2","Type":"ContainerStarted","Data":"56eda0fff55c99f5a556895ddc6049ccc45ae0f6d510bb9d0590c880b792c4e7"}
Sep 07 09:00:09.863334 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.863312    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerStarted","Data":"92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5"}
Sep 07 09:00:09.863448 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.863378    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerStarted","Data":"1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5"}
Sep 07 09:00:09.863448 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.863419    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerStarted","Data":"d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026"}
Sep 07 09:00:09.863560 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.863457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerStarted","Data":"f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e"}
Sep 07 09:00:09.863560 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.863497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerStarted","Data":"e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55"}
Sep 07 09:00:09.879596 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:09.879545    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5" podStartSLOduration=2.040746144 podStartE2EDuration="4.879530397s" totalImagesPullingTime="2.838784253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:09.878086526 +0000 UTC m=+67.847257363" watchObservedRunningTime="2026-09-07 09:00:09.879530397 +0000 UTC m=+67.848701001"
Sep 07 09:00:10.877101 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:10.877037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerStarted","Data":"2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44"}
Sep 07 09:00:10.880534 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:10.880441    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:10.882208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:10.881953    2563 patch_prober.go:28] interesting pod/monitoring-plugin-85bbbbbf4-rq2rh container/monitoring-plugin namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.13:9443/health\": dial tcp 10.134.0.13:9443: connect: connection refused" start-of-body=
Sep 07 09:00:10.882208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:10.881997    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh" podUID="c3b71592-f725-4636-8ada-7e6a51564fcc" containerName="monitoring-plugin" probeResult="failure" output="Get \"https://10.134.0.13:9443/health\": dial tcp 10.134.0.13:9443: connect: connection refused"
Sep 07 09:00:10.908600 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:10.907626    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.822527935 podStartE2EDuration="6.907605033s" totalImagesPullingTime="3.068315097s" totalInitContainerRuntime="16.762001ms" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:10.904153226 +0000 UTC m=+68.873323832" watchObservedRunningTime="2026-09-07 09:00:10.907605033 +0000 UTC m=+68.876775635"
Sep 07 09:00:10.922229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:10.922181    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh" podStartSLOduration=1.6401471029999999 podStartE2EDuration="3.922161996s" totalImagesPullingTime="2.282014893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:10.920634669 +0000 UTC m=+68.889805275" watchObservedRunningTime="2026-09-07 09:00:10.922161996 +0000 UTC m=+68.891332597"
Sep 07 09:00:11.022791 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.022762    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 09:00:11.028412 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:00:11.028385    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16f08a4_dd4d_47b3_878f_fd046e6e847d.slice/crio-a0b77cc3d6dec88958337e8e3dddad2cc3367156c1f86794f46d8705fdc5c510 WatchSource:0}: Error finding container a0b77cc3d6dec88958337e8e3dddad2cc3367156c1f86794f46d8705fdc5c510: Status 404 returned error can't find the container with id a0b77cc3d6dec88958337e8e3dddad2cc3367156c1f86794f46d8705fdc5c510
Sep 07 09:00:11.886431 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.886392    2563 generic.go:356] "Generic (PLEG): container finished" podID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerID="5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da" exitCode=0
Sep 07 09:00:11.886828 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.886489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerDied","Data":"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"}
Sep 07 09:00:11.886828 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.886585    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerStarted","Data":"a0b77cc3d6dec88958337e8e3dddad2cc3367156c1f86794f46d8705fdc5c510"}
Sep 07 09:00:11.889403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.889315    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bmllm" event={"ID":"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5","Type":"ContainerStarted","Data":"d725a1fb18718674d00d18d006dcf1c986e15d98e7c64e806cb7c76bf97a12f5"}
Sep 07 09:00:11.889403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.889364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bmllm" event={"ID":"79f55277-100d-44e8-b8e4-a1ca8c9ed7e5","Type":"ContainerStarted","Data":"edbeb1bde62cd63dae3fa105c813f7e2a5246d1cdc736826ef51884841036580"}
Sep 07 09:00:11.892632 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.892572    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh" event={"ID":"c3b71592-f725-4636-8ada-7e6a51564fcc","Type":"ContainerStarted","Data":"63e2fef2e3c2788e3399b1ed8eda43ad870ac61041d74aa589924a2fe923f9cb"}
Sep 07 09:00:11.895407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.895377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f" event={"ID":"73de73c6-b550-41e0-a5d4-8e921102b155","Type":"ContainerStarted","Data":"c02a08ea4ba0a03efcbd89391c831d64cdfb2d50ec69388aee244c847d2d0222"}
Sep 07 09:00:11.897775 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.897754    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-rq2rh"
Sep 07 09:00:11.929340 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.929276    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bmllm" podStartSLOduration=66.866090603 podStartE2EDuration="1m8.929258233s" totalImagesPullingTime="2.06316763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:11.928887629 +0000 UTC m=+69.898058234" watchObservedRunningTime="2026-09-07 09:00:11.929258233 +0000 UTC m=+69.898428840"
Sep 07 09:00:11.960264 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:11.960036    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f" podStartSLOduration=2.30373842 podStartE2EDuration="4.960018567s" totalImagesPullingTime="2.656280147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:11.958812128 +0000 UTC m=+69.927982732" watchObservedRunningTime="2026-09-07 09:00:11.960018567 +0000 UTC m=+69.929189176"
Sep 07 09:00:14.776525 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:14.776489    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d4c6c4978-zbzln"
Sep 07 09:00:15.886821 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:15.886788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-789cbcdfc6-drpk5"
Sep 07 09:00:16.919175 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.919140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xpfh7" event={"ID":"da4c2b44-97ef-47ec-812b-6f3bb1b5c536","Type":"ContainerStarted","Data":"81b556c0dff971d3fb898ada20bb44d3be429b8cc278fc7af3ee1be2024a427f"}
Sep 07 09:00:16.920706 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.920679    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f7bgm" event={"ID":"6c610c6c-2b93-4fcd-b3aa-9d971fe6c9a2","Type":"ContainerStarted","Data":"7bbb96d6e0c181996f64ba81a21c34d48db805356e713656699e8f72fd76c198"}
Sep 07 09:00:16.920847 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.920746    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 09:00:16.924019 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.923994    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerStarted","Data":"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"}
Sep 07 09:00:16.924139 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.924048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerStarted","Data":"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"}
Sep 07 09:00:16.924139 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.924076    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerStarted","Data":"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"}
Sep 07 09:00:16.924139 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.924106    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerStarted","Data":"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"}
Sep 07 09:00:16.924139 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.924130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerStarted","Data":"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"}
Sep 07 09:00:16.924271 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.924154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerStarted","Data":"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"}
Sep 07 09:00:16.935120 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.935078    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xpfh7" podStartSLOduration=66.688383876 podStartE2EDuration="1m13.935065192s" totalImagesPullingTime="7.246681316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:16.933916089 +0000 UTC m=+74.903086700" watchObservedRunningTime="2026-09-07 09:00:16.935065192 +0000 UTC m=+74.904235796"
Sep 07 09:00:16.947994 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.947954    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f7bgm" podStartSLOduration=67.247385759 podStartE2EDuration="1m13.947941779s" totalImagesPullingTime="6.70055602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:16.946907081 +0000 UTC m=+74.916077686" watchObservedRunningTime="2026-09-07 09:00:16.947941779 +0000 UTC m=+74.917112384"
Sep 07 09:00:16.974219 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:16.974167    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.874776277 podStartE2EDuration="7.974149073s" totalImagesPullingTime="4.080565465s" totalInitContainerRuntime="18.807331ms" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:16.971672068 +0000 UTC m=+74.940842674" watchObservedRunningTime="2026-09-07 09:00:16.974149073 +0000 UTC m=+74.943319680"
Sep 07 09:00:19.752817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:19.752776    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:00:27.781277 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:27.781167    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:27.781277 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:27.781227    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:42.907662 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:42.907626    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jvwmj_bb78a497-2227-45ef-bd86-fc1cdece6bae/serve-healthcheck-canary/0.log"
Sep 07 09:00:47.788854 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:47.788819    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:47.794663 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:47.794632    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79b8f874c5-5dq4f"
Sep 07 09:00:47.931158 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:00:47.931128    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f7bgm"
Sep 07 09:01:09.752106 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:09.752059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:09.773587 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:09.773557    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:10.155185 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:10.155098    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:23.402623 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:23.402576    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 09:01:23.403212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:23.402921    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="alertmanager" containerID="cri-o://e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55" gracePeriod=120
Sep 07 09:01:23.403212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:23.402987    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy-metric" containerID="cri-o://92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5" gracePeriod=120
Sep 07 09:01:23.403212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:23.403026    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="config-reloader" containerID="cri-o://f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e" gracePeriod=120
Sep 07 09:01:23.403212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:23.403072    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="prom-label-proxy" containerID="cri-o://2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44" gracePeriod=120
Sep 07 09:01:23.403212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:23.402985    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy-web" containerID="cri-o://d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026" gracePeriod=120
Sep 07 09:01:23.403212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:23.403035    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy" containerID="cri-o://1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5" gracePeriod=120
Sep 07 09:01:24.201453 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201417    2563 generic.go:356] "Generic (PLEG): container finished" podID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerID="2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44" exitCode=0
Sep 07 09:01:24.201453 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201444    2563 generic.go:356] "Generic (PLEG): container finished" podID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerID="92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5" exitCode=0
Sep 07 09:01:24.201453 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201453    2563 generic.go:356] "Generic (PLEG): container finished" podID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerID="1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5" exitCode=0
Sep 07 09:01:24.201453 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201461    2563 generic.go:356] "Generic (PLEG): container finished" podID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerID="f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e" exitCode=0
Sep 07 09:01:24.201758 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201472    2563 generic.go:356] "Generic (PLEG): container finished" podID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerID="e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55" exitCode=0
Sep 07 09:01:24.201758 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerDied","Data":"2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44"}
Sep 07 09:01:24.201758 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerDied","Data":"92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5"}
Sep 07 09:01:24.201758 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerDied","Data":"1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5"}
Sep 07 09:01:24.201758 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerDied","Data":"f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e"}
Sep 07 09:01:24.201758 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.201639    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerDied","Data":"e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55"}
Sep 07 09:01:24.636294 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.636265    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:24.771208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771178    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-cluster-tls-config\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771402 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771229    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-tls-assets\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771402 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771254    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-main-db\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771529 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771395    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-volume\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771529 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771437    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771630 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771567    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:01:24.771713 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771682    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-out\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771784 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771735    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-trusted-ca-bundle\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771784 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771766    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-metrics-client-ca\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771792    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-main-tls\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771828    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvthq\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-kube-api-access-hvthq\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.771888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771869    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-web\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.772045 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771904    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.772045 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.771945    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-web-config\") pod \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\" (UID: \"f588f39c-0d9d-4dd3-8ef9-b46a43873260\") "
Sep 07 09:01:24.773203 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.772197    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-main-db\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.773203 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.772523    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:01:24.773203 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.773168    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:01:24.775343 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.775228    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:01:24.776177 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.776144    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:24.776177 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.776166    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-out" (OuterVolumeSpecName: "config-out") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:01:24.776374 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.776301    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-volume" (OuterVolumeSpecName: "config-volume") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:24.776434 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.776369    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:24.776621 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.776593    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-kube-api-access-hvthq" (OuterVolumeSpecName: "kube-api-access-hvthq") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "kube-api-access-hvthq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:01:24.776719 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.776650    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:24.776983 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.776957    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:24.779933 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.779903    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:24.785726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.785701    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-web-config" (OuterVolumeSpecName: "web-config") pod "f588f39c-0d9d-4dd3-8ef9-b46a43873260" (UID: "f588f39c-0d9d-4dd3-8ef9-b46a43873260"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:24.872964 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.872942    2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-cluster-tls-config\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.872964 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.872963    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-tls-assets\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.872974    2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.872983    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.872991    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f588f39c-0d9d-4dd3-8ef9-b46a43873260-config-out\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.872999    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.873007    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f588f39c-0d9d-4dd3-8ef9-b46a43873260-metrics-client-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.873015    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-main-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.873024    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvthq\" (UniqueName: \"kubernetes.io/projected/f588f39c-0d9d-4dd3-8ef9-b46a43873260-kube-api-access-hvthq\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.873050    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.873060    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:24.873078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:24.873069    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f588f39c-0d9d-4dd3-8ef9-b46a43873260-web-config\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:25.208873 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.208805    2563 generic.go:356] "Generic (PLEG): container finished" podID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerID="d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026" exitCode=0
Sep 07 09:01:25.208980 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.208877    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.208980 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.208888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerDied","Data":"d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026"}
Sep 07 09:01:25.208980 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.208962    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f588f39c-0d9d-4dd3-8ef9-b46a43873260","Type":"ContainerDied","Data":"dd75d908f3953a140f7aa1fe5b5dd54293f0b6a1edf609063f4e840cef751d0e"}
Sep 07 09:01:25.209095 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.209007    2563 scope.go:117] "RemoveContainer" containerID="2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44"
Sep 07 09:01:25.217425 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.217406    2563 scope.go:117] "RemoveContainer" containerID="92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5"
Sep 07 09:01:25.224375 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.224355    2563 scope.go:117] "RemoveContainer" containerID="1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5"
Sep 07 09:01:25.233656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.232526    2563 scope.go:117] "RemoveContainer" containerID="d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026"
Sep 07 09:01:25.234827 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.234803    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 09:01:25.238761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.238738    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 09:01:25.242568 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.242552    2563 scope.go:117] "RemoveContainer" containerID="f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e"
Sep 07 09:01:25.249532 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.249518    2563 scope.go:117] "RemoveContainer" containerID="e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55"
Sep 07 09:01:25.256305 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.256274    2563 scope.go:117] "RemoveContainer" containerID="d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2"
Sep 07 09:01:25.262037 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262016    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 09:01:25.262398 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262383    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="init-config-reloader"
Sep 07 09:01:25.262451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262402    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="init-config-reloader"
Sep 07 09:01:25.262451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262415    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="alertmanager"
Sep 07 09:01:25.262451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262424    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="alertmanager"
Sep 07 09:01:25.262451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262434    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="prom-label-proxy"
Sep 07 09:01:25.262451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262442    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="prom-label-proxy"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262465    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy-web"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262475    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy-web"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262493    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy-metric"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262502    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy-metric"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262514    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262522    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262531    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="config-reloader"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262539    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="config-reloader"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262593    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy-web"
Sep 07 09:01:25.262605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262606    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="config-reloader"
Sep 07 09:01:25.262892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262614    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="prom-label-proxy"
Sep 07 09:01:25.262892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262622    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="alertmanager"
Sep 07 09:01:25.262892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262629    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy-metric"
Sep 07 09:01:25.262892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.262639    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" containerName="kube-rbac-proxy"
Sep 07 09:01:25.263765 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.263743    2563 scope.go:117] "RemoveContainer" containerID="2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44"
Sep 07 09:01:25.264099 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:25.264067    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44\": container with ID starting with 2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44 not found: ID does not exist" containerID="2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44"
Sep 07 09:01:25.264174 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.264105    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44"} err="failed to get container status \"2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44\": rpc error: code = NotFound desc = could not find container \"2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44\": container with ID starting with 2d4c684622dd1976bf335dcc2c43db317b1b67525329cc87ad797ba759fecf44 not found: ID does not exist"
Sep 07 09:01:25.264174 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.264137    2563 scope.go:117] "RemoveContainer" containerID="92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5"
Sep 07 09:01:25.264420 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:25.264402    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5\": container with ID starting with 92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5 not found: ID does not exist" containerID="92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5"
Sep 07 09:01:25.264475 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.264426    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5"} err="failed to get container status \"92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5\": rpc error: code = NotFound desc = could not find container \"92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5\": container with ID starting with 92642fcacc4dc80ab43835c22c38e4d63aba3cd3b43e0e2fa2caab6bae34bcb5 not found: ID does not exist"
Sep 07 09:01:25.264475 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.264442    2563 scope.go:117] "RemoveContainer" containerID="1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5"
Sep 07 09:01:25.264726 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:25.264705    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5\": container with ID starting with 1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5 not found: ID does not exist" containerID="1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5"
Sep 07 09:01:25.264820 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.264730    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5"} err="failed to get container status \"1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5\": rpc error: code = NotFound desc = could not find container \"1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5\": container with ID starting with 1b85b0f28f24b5095e5eb3762046ab2bf4528c52e6cea67bcdaafaf996efe2d5 not found: ID does not exist"
Sep 07 09:01:25.264820 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.264744    2563 scope.go:117] "RemoveContainer" containerID="d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026"
Sep 07 09:01:25.265004 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:25.264981    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026\": container with ID starting with d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026 not found: ID does not exist" containerID="d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026"
Sep 07 09:01:25.265042 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.265013    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026"} err="failed to get container status \"d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026\": rpc error: code = NotFound desc = could not find container \"d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026\": container with ID starting with d6fb474980c1bf1069313f7bbfc31abbc2e48dcddb7be17c3de77ede7ad3f026 not found: ID does not exist"
Sep 07 09:01:25.265042 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.265033    2563 scope.go:117] "RemoveContainer" containerID="f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e"
Sep 07 09:01:25.265301 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:25.265261    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e\": container with ID starting with f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e not found: ID does not exist" containerID="f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e"
Sep 07 09:01:25.265365 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.265301    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e"} err="failed to get container status \"f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e\": rpc error: code = NotFound desc = could not find container \"f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e\": container with ID starting with f8c675667d2fa8d545600509ee12e220680cb04cf7723d2720a99aa4d157009e not found: ID does not exist"
Sep 07 09:01:25.265365 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.265321    2563 scope.go:117] "RemoveContainer" containerID="e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55"
Sep 07 09:01:25.265535 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:25.265518    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55\": container with ID starting with e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55 not found: ID does not exist" containerID="e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55"
Sep 07 09:01:25.265596 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.265542    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55"} err="failed to get container status \"e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55\": rpc error: code = NotFound desc = could not find container \"e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55\": container with ID starting with e55b7abc55df40e41dcb2ba1fd3f313ce6583b075324e576df816197ecdf2f55 not found: ID does not exist"
Sep 07 09:01:25.265596 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.265561    2563 scope.go:117] "RemoveContainer" containerID="d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2"
Sep 07 09:01:25.265845 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:25.265824    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2\": container with ID starting with d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2 not found: ID does not exist" containerID="d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2"
Sep 07 09:01:25.265887 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.265851    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2"} err="failed to get container status \"d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2\": rpc error: code = NotFound desc = could not find container \"d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2\": container with ID starting with d7b4b740d59703e630e76b0e358a6372e7b7f50e4ac761a0666e96843f878ed2 not found: ID does not exist"
Sep 07 09:01:25.266810 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.266793    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.269175 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.269160    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 07 09:01:25.269295 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.269266    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 07 09:01:25.269539 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.269523    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 07 09:01:25.269673 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.269653    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 07 09:01:25.269745 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.269711    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 07 09:01:25.269745 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.269728    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 07 09:01:25.269946 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.269926    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 07 09:01:25.270071 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.270055    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fddj5\""
Sep 07 09:01:25.270220 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.270190    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 07 09:01:25.282238 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.282214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 09:01:25.282367 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.282313    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 07 09:01:25.382255 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bfd83436-230a-4e51-b199-ca5640e31222-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382269    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bfd83436-230a-4e51-b199-ca5640e31222-config-out\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bfd83436-230a-4e51-b199-ca5640e31222-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd83436-230a-4e51-b199-ca5640e31222-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-config-volume\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58s6c\" (UniqueName: \"kubernetes.io/projected/bfd83436-230a-4e51-b199-ca5640e31222-kube-api-access-58s6c\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.382646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-web-config\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.383013 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.382665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd83436-230a-4e51-b199-ca5640e31222-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483350 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483350 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483332    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483350 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd83436-230a-4e51-b199-ca5640e31222-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-config-volume\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483389    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58s6c\" (UniqueName: \"kubernetes.io/projected/bfd83436-230a-4e51-b199-ca5640e31222-kube-api-access-58s6c\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-web-config\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd83436-230a-4e51-b199-ca5640e31222-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bfd83436-230a-4e51-b199-ca5640e31222-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bfd83436-230a-4e51-b199-ca5640e31222-config-out\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.483561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.484017 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bfd83436-230a-4e51-b199-ca5640e31222-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.484017 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.483933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bfd83436-230a-4e51-b199-ca5640e31222-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.484244 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.484215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd83436-230a-4e51-b199-ca5640e31222-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.485082 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.485053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd83436-230a-4e51-b199-ca5640e31222-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.486303 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.486243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bfd83436-230a-4e51-b199-ca5640e31222-config-out\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.486384 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.486365    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-config-volume\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.486444 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.486398    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.486675 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.486645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.486800 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.486769    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-web-config\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.486881 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.486862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.487141 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.487124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.487558 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.487537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfd83436-230a-4e51-b199-ca5640e31222-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.488294 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.488264    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bfd83436-230a-4e51-b199-ca5640e31222-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.491230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.491206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58s6c\" (UniqueName: \"kubernetes.io/projected/bfd83436-230a-4e51-b199-ca5640e31222-kube-api-access-58s6c\") pod \"alertmanager-main-0\" (UID: \"bfd83436-230a-4e51-b199-ca5640e31222\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.580407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.580378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 09:01:25.855165 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:25.855139    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 09:01:25.857024 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:01:25.856997    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd83436_230a_4e51_b199_ca5640e31222.slice/crio-7cc7c0b1c7bc67d6efcb58f1acf3e63ef445fc67e07a66f482d52e71f2277dc0 WatchSource:0}: Error finding container 7cc7c0b1c7bc67d6efcb58f1acf3e63ef445fc67e07a66f482d52e71f2277dc0: Status 404 returned error can't find the container with id 7cc7c0b1c7bc67d6efcb58f1acf3e63ef445fc67e07a66f482d52e71f2277dc0
Sep 07 09:01:26.215030 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:26.214955    2563 generic.go:356] "Generic (PLEG): container finished" podID="bfd83436-230a-4e51-b199-ca5640e31222" containerID="6413d5f9d0450bd926983b487bb247af08dc44490a06b93a3283818169d7aaf7" exitCode=0
Sep 07 09:01:26.215151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:26.215062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfd83436-230a-4e51-b199-ca5640e31222","Type":"ContainerDied","Data":"6413d5f9d0450bd926983b487bb247af08dc44490a06b93a3283818169d7aaf7"}
Sep 07 09:01:26.215151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:26.215144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfd83436-230a-4e51-b199-ca5640e31222","Type":"ContainerStarted","Data":"7cc7c0b1c7bc67d6efcb58f1acf3e63ef445fc67e07a66f482d52e71f2277dc0"}
Sep 07 09:01:26.464315 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:26.464272    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f588f39c-0d9d-4dd3-8ef9-b46a43873260" path="/var/lib/kubelet/pods/f588f39c-0d9d-4dd3-8ef9-b46a43873260/volumes"
Sep 07 09:01:27.223267 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.223230    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfd83436-230a-4e51-b199-ca5640e31222","Type":"ContainerStarted","Data":"1428671bd2eff3ce4afef8036b765e6ed677cd67c98b05d03632b4a45293830d"}
Sep 07 09:01:27.223628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.223320    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfd83436-230a-4e51-b199-ca5640e31222","Type":"ContainerStarted","Data":"afa327889f525468657b86847d888659e2d48aae4b07d1a25dc63ad45a65baec"}
Sep 07 09:01:27.223628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.223347    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfd83436-230a-4e51-b199-ca5640e31222","Type":"ContainerStarted","Data":"cc4e71daa53414302b3f98924d16f2d8c35c3505f205d29c544b7ec3df661c80"}
Sep 07 09:01:27.223628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.223408    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfd83436-230a-4e51-b199-ca5640e31222","Type":"ContainerStarted","Data":"eb981cf4412788140d1c7fcc56fc353f322fcecc34c095183295f87eb6639915"}
Sep 07 09:01:27.223628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.223434    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfd83436-230a-4e51-b199-ca5640e31222","Type":"ContainerStarted","Data":"060c02ee55bfb9bf8129dacf907c9024676553be44eb5bedc43ee154b55cb5d7"}
Sep 07 09:01:27.223628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.223458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bfd83436-230a-4e51-b199-ca5640e31222","Type":"ContainerStarted","Data":"e02fbc14bbdae832ff509a748ca848be7f3df92f8d44efa2a682682ba6974938"}
Sep 07 09:01:27.250326 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.250258    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.230642542 podStartE2EDuration="2.250243905s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.601363ms" isStatefulPod=false podCreationTimestamp="2026-09-07 09:01:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:01:27.247821564 +0000 UTC m=+145.216992176" watchObservedRunningTime="2026-09-07 09:01:27.250243905 +0000 UTC m=+145.219414509"
Sep 07 09:01:27.713574 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.713535    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 09:01:27.713919 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.713864    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="prometheus" containerID="cri-o://f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672" gracePeriod=600
Sep 07 09:01:27.713919 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.713886    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy" containerID="cri-o://0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2" gracePeriod=600
Sep 07 09:01:27.714122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.713931    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="thanos-sidecar" containerID="cri-o://049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3" gracePeriod=600
Sep 07 09:01:27.714122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.713962    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="config-reloader" containerID="cri-o://47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314" gracePeriod=600
Sep 07 09:01:27.714122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.713917    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy-web" containerID="cri-o://7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb" gracePeriod=600
Sep 07 09:01:27.714122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.713929    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe" gracePeriod=600
Sep 07 09:01:27.941435 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:27.941409    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.003771 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.003746    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.003922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.003779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.003922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.003798    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-tls-assets\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.003922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.003815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-trusted-ca-bundle\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.003922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.003844    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-kubelet-serving-ca-bundle\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.003922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.003871    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.003922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.003910    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-grpc-tls\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.003967    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-thanos-prometheus-http-client-file\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004005    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-web-config\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004033    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-tls\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004059    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-metrics-client-ca\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004098    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-serving-certs-ca-bundle\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004127    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-1\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004164    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wtp6\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-kube-api-access-9wtp6\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004200    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-2\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004654 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004240    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-metrics-client-certs\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004654 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004277    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-0\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004654 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004334    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-db\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004654 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004358    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-kube-rbac-proxy\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.004654 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004393    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config-out\") pod \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\" (UID: \"f16f08a4-dd4d-47b3-878f-fd046e6e847d\") "
Sep 07 09:01:28.005427 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004233    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:01:28.005526 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004472    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:01:28.005526 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.004831    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:01:28.005526 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.005224    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:01:28.006206 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.006180    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:01:28.007136 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.007106    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:01:28.007746 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.007696    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:01:28.008391 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.008362    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:01:28.008391 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.008368    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:28.008709 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.008681    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config" (OuterVolumeSpecName: "config") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:28.009058 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.009024    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config-out" (OuterVolumeSpecName: "config-out") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:01:28.009138 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.009063    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:01:28.010426 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.010368    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:28.010534 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.010440    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-kube-api-access-9wtp6" (OuterVolumeSpecName: "kube-api-access-9wtp6") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "kube-api-access-9wtp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:01:28.010534 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.010489    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:28.010695 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.010576    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:28.010748 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.010722    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:28.010834 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.010816    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:28.011953 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.011931    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:28.020177 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.020153    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-web-config" (OuterVolumeSpecName: "web-config") pod "f16f08a4-dd4d-47b3-878f-fd046e6e847d" (UID: "f16f08a4-dd4d-47b3-878f-fd046e6e847d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:01:28.105787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105757    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105781    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105791    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-grpc-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105800    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105808    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-web-config\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105816    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105824    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-metrics-client-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105834    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105842    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105850    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wtp6\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-kube-api-access-9wtp6\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105858    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105866    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-metrics-client-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105874    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105883    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-k8s-db\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105891    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-kube-rbac-proxy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105900    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config-out\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105908    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105916    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f16f08a4-dd4d-47b3-878f-fd046e6e847d-config\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105923    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f16f08a4-dd4d-47b3-878f-fd046e6e847d-tls-assets\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.105944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.105931    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f16f08a4-dd4d-47b3-878f-fd046e6e847d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:01:28.234986 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.234954    2563 generic.go:356] "Generic (PLEG): container finished" podID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerID="8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe" exitCode=0
Sep 07 09:01:28.234986 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.234979    2563 generic.go:356] "Generic (PLEG): container finished" podID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerID="0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2" exitCode=0
Sep 07 09:01:28.234986 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.234985    2563 generic.go:356] "Generic (PLEG): container finished" podID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerID="7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb" exitCode=0
Sep 07 09:01:28.234986 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.234991    2563 generic.go:356] "Generic (PLEG): container finished" podID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerID="049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3" exitCode=0
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.234997    2563 generic.go:356] "Generic (PLEG): container finished" podID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerID="47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314" exitCode=0
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235002    2563 generic.go:356] "Generic (PLEG): container finished" podID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerID="f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672" exitCode=0
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235032    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerDied","Data":"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"}
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerDied","Data":"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"}
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerDied","Data":"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"}
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerDied","Data":"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"}
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235201    2563 scope.go:117] "RemoveContainer" containerID="8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerDied","Data":"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"}
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerDied","Data":"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"}
Sep 07 09:01:28.235446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.235324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f16f08a4-dd4d-47b3-878f-fd046e6e847d","Type":"ContainerDied","Data":"a0b77cc3d6dec88958337e8e3dddad2cc3367156c1f86794f46d8705fdc5c510"}
Sep 07 09:01:28.245564 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.245548    2563 scope.go:117] "RemoveContainer" containerID="0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"
Sep 07 09:01:28.252272 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.252256    2563 scope.go:117] "RemoveContainer" containerID="7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"
Sep 07 09:01:28.258896 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.258877    2563 scope.go:117] "RemoveContainer" containerID="049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"
Sep 07 09:01:28.259674 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.259650    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 09:01:28.263991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.263963    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 09:01:28.265518 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.265504    2563 scope.go:117] "RemoveContainer" containerID="47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"
Sep 07 09:01:28.271996 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.271980    2563 scope.go:117] "RemoveContainer" containerID="f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"
Sep 07 09:01:28.278974 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.278957    2563 scope.go:117] "RemoveContainer" containerID="5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"
Sep 07 09:01:28.285050 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.285035    2563 scope.go:117] "RemoveContainer" containerID="8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"
Sep 07 09:01:28.285312 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:28.285291    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": container with ID starting with 8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe not found: ID does not exist" containerID="8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"
Sep 07 09:01:28.285359 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.285322    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"} err="failed to get container status \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": rpc error: code = NotFound desc = could not find container \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": container with ID starting with 8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe not found: ID does not exist"
Sep 07 09:01:28.285359 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.285338    2563 scope.go:117] "RemoveContainer" containerID="0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"
Sep 07 09:01:28.285600 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:28.285581    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": container with ID starting with 0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2 not found: ID does not exist" containerID="0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"
Sep 07 09:01:28.285658 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.285606    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"} err="failed to get container status \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": rpc error: code = NotFound desc = could not find container \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": container with ID starting with 0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2 not found: ID does not exist"
Sep 07 09:01:28.285658 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.285623    2563 scope.go:117] "RemoveContainer" containerID="7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"
Sep 07 09:01:28.285819 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:28.285803    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": container with ID starting with 7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb not found: ID does not exist" containerID="7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"
Sep 07 09:01:28.285849 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.285823    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"} err="failed to get container status \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": rpc error: code = NotFound desc = could not find container \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": container with ID starting with 7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb not found: ID does not exist"
Sep 07 09:01:28.285849 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.285835    2563 scope.go:117] "RemoveContainer" containerID="049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"
Sep 07 09:01:28.286057 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:28.286042    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": container with ID starting with 049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3 not found: ID does not exist" containerID="049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"
Sep 07 09:01:28.286092 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.286060    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"} err="failed to get container status \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": rpc error: code = NotFound desc = could not find container \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": container with ID starting with 049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3 not found: ID does not exist"
Sep 07 09:01:28.286092 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.286072    2563 scope.go:117] "RemoveContainer" containerID="47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"
Sep 07 09:01:28.286328 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:28.286276    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": container with ID starting with 47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314 not found: ID does not exist" containerID="47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"
Sep 07 09:01:28.286374 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.286333    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"} err="failed to get container status \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": rpc error: code = NotFound desc = could not find container \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": container with ID starting with 47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314 not found: ID does not exist"
Sep 07 09:01:28.286374 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.286349    2563 scope.go:117] "RemoveContainer" containerID="f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"
Sep 07 09:01:28.286741 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:28.286722    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": container with ID starting with f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672 not found: ID does not exist" containerID="f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"
Sep 07 09:01:28.286850 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.286748    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"} err="failed to get container status \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": rpc error: code = NotFound desc = could not find container \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": container with ID starting with f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672 not found: ID does not exist"
Sep 07 09:01:28.286850 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.286768    2563 scope.go:117] "RemoveContainer" containerID="5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"
Sep 07 09:01:28.287100 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:01:28.287081    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": container with ID starting with 5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da not found: ID does not exist" containerID="5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"
Sep 07 09:01:28.287168 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.287105    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"} err="failed to get container status \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": rpc error: code = NotFound desc = could not find container \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": container with ID starting with 5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da not found: ID does not exist"
Sep 07 09:01:28.287168 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.287125    2563 scope.go:117] "RemoveContainer" containerID="8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"
Sep 07 09:01:28.287442 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.287421    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"} err="failed to get container status \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": rpc error: code = NotFound desc = could not find container \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": container with ID starting with 8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe not found: ID does not exist"
Sep 07 09:01:28.287499 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.287444    2563 scope.go:117] "RemoveContainer" containerID="0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"
Sep 07 09:01:28.288085 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288062    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"} err="failed to get container status \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": rpc error: code = NotFound desc = could not find container \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": container with ID starting with 0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2 not found: ID does not exist"
Sep 07 09:01:28.288085 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288084    2563 scope.go:117] "RemoveContainer" containerID="7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"
Sep 07 09:01:28.288334 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288314    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"} err="failed to get container status \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": rpc error: code = NotFound desc = could not find container \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": container with ID starting with 7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb not found: ID does not exist"
Sep 07 09:01:28.288334 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288333    2563 scope.go:117] "RemoveContainer" containerID="049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"
Sep 07 09:01:28.288448 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288410    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 09:01:28.288550 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288535    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"} err="failed to get container status \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": rpc error: code = NotFound desc = could not find container \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": container with ID starting with 049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3 not found: ID does not exist"
Sep 07 09:01:28.288602 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288550    2563 scope.go:117] "RemoveContainer" containerID="47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"
Sep 07 09:01:28.288686 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288673    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy-web"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288688    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy-web"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288699    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy-thanos"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288706    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy-thanos"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288718    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="init-config-reloader"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288726    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="init-config-reloader"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288737    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288744    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288751    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="thanos-sidecar"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288756    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="thanos-sidecar"
Sep 07 09:01:28.288761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288764    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="prometheus"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288770    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="prometheus"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288776    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="config-reloader"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288781    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="config-reloader"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288792    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"} err="failed to get container status \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": rpc error: code = NotFound desc = could not find container \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": container with ID starting with 47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314 not found: ID does not exist"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288807    2563 scope.go:117] "RemoveContainer" containerID="f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288826    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288834    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="config-reloader"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288840    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy-thanos"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288844    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="thanos-sidecar"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288849    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="kube-rbac-proxy-web"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.288856    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" containerName="prometheus"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.289038    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"} err="failed to get container status \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": rpc error: code = NotFound desc = could not find container \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": container with ID starting with f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672 not found: ID does not exist"
Sep 07 09:01:28.289077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.289063    2563 scope.go:117] "RemoveContainer" containerID="5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"
Sep 07 09:01:28.289511 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.289329    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"} err="failed to get container status \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": rpc error: code = NotFound desc = could not find container \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": container with ID starting with 5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da not found: ID does not exist"
Sep 07 09:01:28.289511 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.289344    2563 scope.go:117] "RemoveContainer" containerID="8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"
Sep 07 09:01:28.289580 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.289545    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"} err="failed to get container status \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": rpc error: code = NotFound desc = could not find container \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": container with ID starting with 8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe not found: ID does not exist"
Sep 07 09:01:28.289580 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.289558    2563 scope.go:117] "RemoveContainer" containerID="0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"
Sep 07 09:01:28.289782 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.289766    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"} err="failed to get container status \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": rpc error: code = NotFound desc = could not find container \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": container with ID starting with 0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2 not found: ID does not exist"
Sep 07 09:01:28.289822 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.289782    2563 scope.go:117] "RemoveContainer" containerID="7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"
Sep 07 09:01:28.289999 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.289982    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"} err="failed to get container status \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": rpc error: code = NotFound desc = could not find container \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": container with ID starting with 7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb not found: ID does not exist"
Sep 07 09:01:28.290043 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.290000    2563 scope.go:117] "RemoveContainer" containerID="049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"
Sep 07 09:01:28.290213 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.290194    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"} err="failed to get container status \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": rpc error: code = NotFound desc = could not find container \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": container with ID starting with 049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3 not found: ID does not exist"
Sep 07 09:01:28.290304 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.290214    2563 scope.go:117] "RemoveContainer" containerID="47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"
Sep 07 09:01:28.290448 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.290430    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"} err="failed to get container status \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": rpc error: code = NotFound desc = could not find container \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": container with ID starting with 47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314 not found: ID does not exist"
Sep 07 09:01:28.290487 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.290449    2563 scope.go:117] "RemoveContainer" containerID="f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"
Sep 07 09:01:28.290651 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.290631    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"} err="failed to get container status \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": rpc error: code = NotFound desc = could not find container \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": container with ID starting with f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672 not found: ID does not exist"
Sep 07 09:01:28.290690 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.290652    2563 scope.go:117] "RemoveContainer" containerID="5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"
Sep 07 09:01:28.290854 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.290836    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"} err="failed to get container status \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": rpc error: code = NotFound desc = could not find container \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": container with ID starting with 5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da not found: ID does not exist"
Sep 07 09:01:28.290916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.290856    2563 scope.go:117] "RemoveContainer" containerID="8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"
Sep 07 09:01:28.291115 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.291094    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"} err="failed to get container status \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": rpc error: code = NotFound desc = could not find container \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": container with ID starting with 8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe not found: ID does not exist"
Sep 07 09:01:28.291156 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.291116    2563 scope.go:117] "RemoveContainer" containerID="0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"
Sep 07 09:01:28.291360 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.291342    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"} err="failed to get container status \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": rpc error: code = NotFound desc = could not find container \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": container with ID starting with 0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2 not found: ID does not exist"
Sep 07 09:01:28.291405 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.291361    2563 scope.go:117] "RemoveContainer" containerID="7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"
Sep 07 09:01:28.291582 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.291565    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"} err="failed to get container status \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": rpc error: code = NotFound desc = could not find container \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": container with ID starting with 7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb not found: ID does not exist"
Sep 07 09:01:28.291619 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.291584    2563 scope.go:117] "RemoveContainer" containerID="049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"
Sep 07 09:01:28.291813 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.291793    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"} err="failed to get container status \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": rpc error: code = NotFound desc = could not find container \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": container with ID starting with 049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3 not found: ID does not exist"
Sep 07 09:01:28.291866 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.291813    2563 scope.go:117] "RemoveContainer" containerID="47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"
Sep 07 09:01:28.292018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.292003    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"} err="failed to get container status \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": rpc error: code = NotFound desc = could not find container \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": container with ID starting with 47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314 not found: ID does not exist"
Sep 07 09:01:28.292061 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.292018    2563 scope.go:117] "RemoveContainer" containerID="f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"
Sep 07 09:01:28.292227 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.292205    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"} err="failed to get container status \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": rpc error: code = NotFound desc = could not find container \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": container with ID starting with f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672 not found: ID does not exist"
Sep 07 09:01:28.292227 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.292225    2563 scope.go:117] "RemoveContainer" containerID="5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"
Sep 07 09:01:28.292459 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.292444    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.292533 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.292501    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"} err="failed to get container status \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": rpc error: code = NotFound desc = could not find container \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": container with ID starting with 5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da not found: ID does not exist"
Sep 07 09:01:28.292533 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.292526    2563 scope.go:117] "RemoveContainer" containerID="8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"
Sep 07 09:01:28.292889 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.292797    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"} err="failed to get container status \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": rpc error: code = NotFound desc = could not find container \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": container with ID starting with 8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe not found: ID does not exist"
Sep 07 09:01:28.292889 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.292827    2563 scope.go:117] "RemoveContainer" containerID="0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"
Sep 07 09:01:28.293156 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.293135    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"} err="failed to get container status \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": rpc error: code = NotFound desc = could not find container \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": container with ID starting with 0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2 not found: ID does not exist"
Sep 07 09:01:28.293156 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.293157    2563 scope.go:117] "RemoveContainer" containerID="7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"
Sep 07 09:01:28.293431 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.293409    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"} err="failed to get container status \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": rpc error: code = NotFound desc = could not find container \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": container with ID starting with 7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb not found: ID does not exist"
Sep 07 09:01:28.293509 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.293433    2563 scope.go:117] "RemoveContainer" containerID="049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"
Sep 07 09:01:28.293705 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.293688    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"} err="failed to get container status \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": rpc error: code = NotFound desc = could not find container \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": container with ID starting with 049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3 not found: ID does not exist"
Sep 07 09:01:28.293760 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.293705    2563 scope.go:117] "RemoveContainer" containerID="47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"
Sep 07 09:01:28.293969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.293945    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"} err="failed to get container status \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": rpc error: code = NotFound desc = could not find container \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": container with ID starting with 47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314 not found: ID does not exist"
Sep 07 09:01:28.294046 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.293971    2563 scope.go:117] "RemoveContainer" containerID="f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"
Sep 07 09:01:28.294266 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.294246    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"} err="failed to get container status \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": rpc error: code = NotFound desc = could not find container \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": container with ID starting with f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672 not found: ID does not exist"
Sep 07 09:01:28.294266 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.294266    2563 scope.go:117] "RemoveContainer" containerID="5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"
Sep 07 09:01:28.294530 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.294509    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"} err="failed to get container status \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": rpc error: code = NotFound desc = could not find container \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": container with ID starting with 5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da not found: ID does not exist"
Sep 07 09:01:28.294578 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.294532    2563 scope.go:117] "RemoveContainer" containerID="8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"
Sep 07 09:01:28.294778 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.294747    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe"} err="failed to get container status \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": rpc error: code = NotFound desc = could not find container \"8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe\": container with ID starting with 8b9b61513b449567b588f1a65fdd94e6c7476e9a1305a1f094fe2cd92b37e1fe not found: ID does not exist"
Sep 07 09:01:28.294778 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.294773    2563 scope.go:117] "RemoveContainer" containerID="0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"
Sep 07 09:01:28.295035 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 07 09:01:28.295110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295058    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2"} err="failed to get container status \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": rpc error: code = NotFound desc = could not find container \"0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2\": container with ID starting with 0dcf1cb6c1565288bab787582287b07340254dec962acd4b100016221360d7f2 not found: ID does not exist"
Sep 07 09:01:28.295110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295080    2563 scope.go:117] "RemoveContainer" containerID="7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"
Sep 07 09:01:28.295110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295102    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 07 09:01:28.295270 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295117    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 07 09:01:28.295270 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295108    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 07 09:01:28.295270 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295160    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 07 09:01:28.295270 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295107    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 07 09:01:28.295270 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295239    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 07 09:01:28.295598 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295537    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb"} err="failed to get container status \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": rpc error: code = NotFound desc = could not find container \"7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb\": container with ID starting with 7d74bbed9d615df17282a691c7d027599a3b416a4d8aab0ae5d788f94ea060eb not found: ID does not exist"
Sep 07 09:01:28.295598 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295565    2563 scope.go:117] "RemoveContainer" containerID="049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"
Sep 07 09:01:28.295598 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295569    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 07 09:01:28.295817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295633    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 07 09:01:28.295817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295690    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 07 09:01:28.295817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295633    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 07 09:01:28.295817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295634    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-85wrm\""
Sep 07 09:01:28.295817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295765    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 07 09:01:28.296154 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295866    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3"} err="failed to get container status \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": rpc error: code = NotFound desc = could not find container \"049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3\": container with ID starting with 049811c3288ee4f9220137f77e14c3d6dba0c85ec9b90fae51411d03f903e1c3 not found: ID does not exist"
Sep 07 09:01:28.296154 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.295887    2563 scope.go:117] "RemoveContainer" containerID="47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"
Sep 07 09:01:28.296154 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.296114    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1rchugldo3igm\""
Sep 07 09:01:28.296154 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.296126    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314"} err="failed to get container status \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": rpc error: code = NotFound desc = could not find container \"47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314\": container with ID starting with 47a5f02c65478ced1c9a1fd9436a2a2c62221c0100a3e64237d8cf178f70c314 not found: ID does not exist"
Sep 07 09:01:28.296360 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.296157    2563 scope.go:117] "RemoveContainer" containerID="f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"
Sep 07 09:01:28.296433 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.296413    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672"} err="failed to get container status \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": rpc error: code = NotFound desc = could not find container \"f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672\": container with ID starting with f3d064e13a1f1fb0c8beceb538e790a8b5f68d2ad05558773bb46e5f82319672 not found: ID does not exist"
Sep 07 09:01:28.296510 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.296433    2563 scope.go:117] "RemoveContainer" containerID="5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"
Sep 07 09:01:28.296739 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.296707    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da"} err="failed to get container status \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": rpc error: code = NotFound desc = could not find container \"5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da\": container with ID starting with 5c43a57ca25660a2f42a1c56027bec6d892e9f4044f547a863e6d3600d2a57da not found: ID does not exist"
Sep 07 09:01:28.298575 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.298558    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 07 09:01:28.300121 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.300105    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 07 09:01:28.313831 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.313812    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 09:01:28.408347 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5fde8ee7-0e57-4f22-858f-f5889b135e20-config-out\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408347 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5fde8ee7-0e57-4f22-858f-f5889b135e20-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408531 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408531 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408531 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408424    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408531 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408531 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvmd7\" (UniqueName: \"kubernetes.io/projected/5fde8ee7-0e57-4f22-858f-f5889b135e20-kube-api-access-qvmd7\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408531 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408548    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408737    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-config\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-web-config\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.408793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.409068 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408839    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.409068 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.409068 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.408893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.464270 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.464246    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16f08a4-dd4d-47b3-878f-fd046e6e847d" path="/var/lib/kubelet/pods/f16f08a4-dd4d-47b3-878f-fd046e6e847d/volumes"
Sep 07 09:01:28.509736 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.509736 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.509736 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.509936 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvmd7\" (UniqueName: \"kubernetes.io/projected/5fde8ee7-0e57-4f22-858f-f5889b135e20-kube-api-access-qvmd7\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.509936 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.509936 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.509936 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.509936 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.509936 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.509983    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-config\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-web-config\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510144    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5fde8ee7-0e57-4f22-858f-f5889b135e20-config-out\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5fde8ee7-0e57-4f22-858f-f5889b135e20-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510349    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510372    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510900 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.510900 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.510845    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.511430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.511404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.513529 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.513362    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.513529 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.513417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.513529 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.513507    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.513712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.513562    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.513712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.513622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-config\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.513712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.513624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.513908 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.513882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.514131 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.514089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.514213 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.514199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.515482 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.515460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5fde8ee7-0e57-4f22-858f-f5889b135e20-config-out\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.515559 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.515485    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5fde8ee7-0e57-4f22-858f-f5889b135e20-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.515722 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.515702    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5fde8ee7-0e57-4f22-858f-f5889b135e20-web-config\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.516446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.516428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5fde8ee7-0e57-4f22-858f-f5889b135e20-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.518270 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.518254    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvmd7\" (UniqueName: \"kubernetes.io/projected/5fde8ee7-0e57-4f22-858f-f5889b135e20-kube-api-access-qvmd7\") pod \"prometheus-k8s-0\" (UID: \"5fde8ee7-0e57-4f22-858f-f5889b135e20\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.603342 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.603311    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:01:28.878359 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:28.878331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 09:01:28.880524 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:01:28.880488    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fde8ee7_0e57_4f22_858f_f5889b135e20.slice/crio-7e4ddc17d4b8c1a36ee15bb2f2d22fe4a8194cb597e927a866dfd420e2db91a3 WatchSource:0}: Error finding container 7e4ddc17d4b8c1a36ee15bb2f2d22fe4a8194cb597e927a866dfd420e2db91a3: Status 404 returned error can't find the container with id 7e4ddc17d4b8c1a36ee15bb2f2d22fe4a8194cb597e927a866dfd420e2db91a3
Sep 07 09:01:29.241440 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:29.241356    2563 generic.go:356] "Generic (PLEG): container finished" podID="5fde8ee7-0e57-4f22-858f-f5889b135e20" containerID="ade0f5f2e06b16bce8c1195b55228f3008f8c0d03992bac91c489385b87af9a8" exitCode=0
Sep 07 09:01:29.241440 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:29.241423    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5fde8ee7-0e57-4f22-858f-f5889b135e20","Type":"ContainerDied","Data":"ade0f5f2e06b16bce8c1195b55228f3008f8c0d03992bac91c489385b87af9a8"}
Sep 07 09:01:29.241799 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:29.241489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5fde8ee7-0e57-4f22-858f-f5889b135e20","Type":"ContainerStarted","Data":"7e4ddc17d4b8c1a36ee15bb2f2d22fe4a8194cb597e927a866dfd420e2db91a3"}
Sep 07 09:01:30.250270 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:30.250225    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5fde8ee7-0e57-4f22-858f-f5889b135e20","Type":"ContainerStarted","Data":"14aebb2d4c1b854be009e97537bf3de43f82855c014c1a301a714d65a7bed909"}
Sep 07 09:01:30.250789 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:30.250381    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5fde8ee7-0e57-4f22-858f-f5889b135e20","Type":"ContainerStarted","Data":"b5b4ea07fb1f52eaa1f97d0527edd16390d44f7c1e08f50966893c41392b015d"}
Sep 07 09:01:30.250789 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:30.250449    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5fde8ee7-0e57-4f22-858f-f5889b135e20","Type":"ContainerStarted","Data":"7a40aee4d314494fe866207291463d9edd1dd349693006cb157acff5bc4653b7"}
Sep 07 09:01:30.250789 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:30.250494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5fde8ee7-0e57-4f22-858f-f5889b135e20","Type":"ContainerStarted","Data":"a582269eb6ac5fdc75254386c4ee6dd41158d2afa6f574809de9e5d01aa2997e"}
Sep 07 09:01:30.250789 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:30.250544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5fde8ee7-0e57-4f22-858f-f5889b135e20","Type":"ContainerStarted","Data":"98f1f4376ad63cc40d184924fa30030324efed9b609278e1b8159fe57576336a"}
Sep 07 09:01:30.250789 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:30.250589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5fde8ee7-0e57-4f22-858f-f5889b135e20","Type":"ContainerStarted","Data":"b34e421a66b98e277e0fa604ba169f2ed7c0a67f9a24a4b7793a5f3c4069258c"}
Sep 07 09:01:30.280316 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:30.280244    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.26051387 podStartE2EDuration="2.28022426s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.71039ms" isStatefulPod=false podCreationTimestamp="2026-09-07 09:01:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:01:30.277407882 +0000 UTC m=+148.246578488" watchObservedRunningTime="2026-09-07 09:01:30.28022426 +0000 UTC m=+148.249394865"
Sep 07 09:01:33.604332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:01:33.604298    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:02:28.604022 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:02:28.603979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:02:28.618895 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:02:28.618866    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:02:29.495729 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:02:29.495699    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 09:04:02.383732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:02.383706    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 07 09:04:28.320165 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.320123    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"]
Sep 07 09:04:28.322432 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.322409    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.325114 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.325084    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 07 09:04:28.325311 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.325094    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 07 09:04:28.325437 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.325417    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c5k27\""
Sep 07 09:04:28.325489 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.325459    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 07 09:04:28.325557 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.325538    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 07 09:04:28.345046 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.345020    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"]
Sep 07 09:04:28.453089 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.453055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4f294bc4-cd28-44e8-be90-1713cafe27cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.453258 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.453112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f294bc4-cd28-44e8-be90-1713cafe27cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.453258 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.453208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4f294bc4-cd28-44e8-be90-1713cafe27cd-manifests\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.453360 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.453306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st657\" (UniqueName: \"kubernetes.io/projected/4f294bc4-cd28-44e8-be90-1713cafe27cd-kube-api-access-st657\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.453360 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.453344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f294bc4-cd28-44e8-be90-1713cafe27cd-tmp\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.553869 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.553837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4f294bc4-cd28-44e8-be90-1713cafe27cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.554041 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.553892    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f294bc4-cd28-44e8-be90-1713cafe27cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.554041 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.553928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4f294bc4-cd28-44e8-be90-1713cafe27cd-manifests\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.554134 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.554055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st657\" (UniqueName: \"kubernetes.io/projected/4f294bc4-cd28-44e8-be90-1713cafe27cd-kube-api-access-st657\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.554134 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.554098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f294bc4-cd28-44e8-be90-1713cafe27cd-tmp\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.554444 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.554422    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4f294bc4-cd28-44e8-be90-1713cafe27cd-manifests\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.556326 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.556303    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f294bc4-cd28-44e8-be90-1713cafe27cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.556689 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.556667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f294bc4-cd28-44e8-be90-1713cafe27cd-tmp\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.556797 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.556780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4f294bc4-cd28-44e8-be90-1713cafe27cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.575843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.575789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st657\" (UniqueName: \"kubernetes.io/projected/4f294bc4-cd28-44e8-be90-1713cafe27cd-kube-api-access-st657\") pod \"opendatahub-operator-controller-manager-78c59b5944-2bllj\" (UID: \"4f294bc4-cd28-44e8-be90-1713cafe27cd\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.610257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.610227    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"]
Sep 07 09:04:28.612778 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.612758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.615656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.615630    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 07 09:04:28.615656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.615660    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 07 09:04:28.615860 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.615697    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 07 09:04:28.615860 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.615768    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c8bwd\""
Sep 07 09:04:28.616006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.615992    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 07 09:04:28.616145 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.616130    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 07 09:04:28.625301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.625255    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"]
Sep 07 09:04:28.633080 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.633056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:28.755681 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.755649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fef8b845-166b-4e83-aa00-fbdbc49cdea6-cert\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.755681 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.755694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fef8b845-166b-4e83-aa00-fbdbc49cdea6-manager-config\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.755899 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.755717    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2dtj\" (UniqueName: \"kubernetes.io/projected/fef8b845-166b-4e83-aa00-fbdbc49cdea6-kube-api-access-r2dtj\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.755899 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.755842    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fef8b845-166b-4e83-aa00-fbdbc49cdea6-metrics-cert\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.856512 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.856427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fef8b845-166b-4e83-aa00-fbdbc49cdea6-cert\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.856512 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.856471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fef8b845-166b-4e83-aa00-fbdbc49cdea6-manager-config\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.856512 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.856490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2dtj\" (UniqueName: \"kubernetes.io/projected/fef8b845-166b-4e83-aa00-fbdbc49cdea6-kube-api-access-r2dtj\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.856770 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.856611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fef8b845-166b-4e83-aa00-fbdbc49cdea6-metrics-cert\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.857106 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.857084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fef8b845-166b-4e83-aa00-fbdbc49cdea6-manager-config\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.858910 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.858888    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fef8b845-166b-4e83-aa00-fbdbc49cdea6-cert\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.859008 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.858969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fef8b845-166b-4e83-aa00-fbdbc49cdea6-metrics-cert\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.874813 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.874783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2dtj\" (UniqueName: \"kubernetes.io/projected/fef8b845-166b-4e83-aa00-fbdbc49cdea6-kube-api-access-r2dtj\") pod \"lws-controller-manager-94f7b5c54-m9zcl\" (UID: \"fef8b845-166b-4e83-aa00-fbdbc49cdea6\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.919681 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.919648    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"]
Sep 07 09:04:28.923055 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:04:28.923030    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f294bc4_cd28_44e8_be90_1713cafe27cd.slice/crio-de3a836d4577910b20560bea0f801362efa6dc9b22343551b80d97df89e753e1 WatchSource:0}: Error finding container de3a836d4577910b20560bea0f801362efa6dc9b22343551b80d97df89e753e1: Status 404 returned error can't find the container with id de3a836d4577910b20560bea0f801362efa6dc9b22343551b80d97df89e753e1
Sep 07 09:04:28.924119 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.924101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:28.924786 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:28.924771    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:04:29.193788 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:29.193719    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"]
Sep 07 09:04:29.195924 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:04:29.195899    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef8b845_166b_4e83_aa00_fbdbc49cdea6.slice/crio-01d868e26552e58cb2ead223e25766dc4fd9f91b078535c6cd0a9c21451d0ae2 WatchSource:0}: Error finding container 01d868e26552e58cb2ead223e25766dc4fd9f91b078535c6cd0a9c21451d0ae2: Status 404 returned error can't find the container with id 01d868e26552e58cb2ead223e25766dc4fd9f91b078535c6cd0a9c21451d0ae2
Sep 07 09:04:29.919483 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:29.919443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl" event={"ID":"fef8b845-166b-4e83-aa00-fbdbc49cdea6","Type":"ContainerStarted","Data":"01d868e26552e58cb2ead223e25766dc4fd9f91b078535c6cd0a9c21451d0ae2"}
Sep 07 09:04:29.921005 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:29.920980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj" event={"ID":"4f294bc4-cd28-44e8-be90-1713cafe27cd","Type":"ContainerStarted","Data":"de3a836d4577910b20560bea0f801362efa6dc9b22343551b80d97df89e753e1"}
Sep 07 09:04:33.939697 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:33.939659    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj" event={"ID":"4f294bc4-cd28-44e8-be90-1713cafe27cd","Type":"ContainerStarted","Data":"c1bc78d3d3eaeb6e9841a8337e6a9b7ad97343cac360632950e7a2a851b8f574"}
Sep 07 09:04:33.941085 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:33.941058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl" event={"ID":"fef8b845-166b-4e83-aa00-fbdbc49cdea6","Type":"ContainerStarted","Data":"6b0b55992837038dfe9ce673f4e31e4ae40d40d61d28a0fb2d5bb5fcbb02230c"}
Sep 07 09:04:33.941211 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:33.941194    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:33.980321 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:33.980254    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl" podStartSLOduration=2.168192897 podStartE2EDuration="5.980238718s" totalImagesPullingTime="3.812045821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:33.97899615 +0000 UTC m=+331.948166755" watchObservedRunningTime="2026-09-07 09:04:33.980238718 +0000 UTC m=+331.949409323"
Sep 07 09:04:34.949012 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:34.948979    2563 generic.go:356] "Generic (PLEG): container finished" podID="4f294bc4-cd28-44e8-be90-1713cafe27cd" containerID="c1bc78d3d3eaeb6e9841a8337e6a9b7ad97343cac360632950e7a2a851b8f574" exitCode=0
Sep 07 09:04:34.949415 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:34.949063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj" event={"ID":"4f294bc4-cd28-44e8-be90-1713cafe27cd","Type":"ContainerDied","Data":"c1bc78d3d3eaeb6e9841a8337e6a9b7ad97343cac360632950e7a2a851b8f574"}
Sep 07 09:04:35.954862 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:35.954821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj" event={"ID":"4f294bc4-cd28-44e8-be90-1713cafe27cd","Type":"ContainerStarted","Data":"756487ddc2ede386a27ec7d5292dcc37d51e82935cdaad38d11e037719315940"}
Sep 07 09:04:35.955211 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:35.954945    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:35.976241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:35.976188    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj" podStartSLOduration=2.483985219 podStartE2EDuration="7.976169478s" totalImagesPullingTime="4.413936068s" totalInitContainerRuntime="1.078248191s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:35.973768378 +0000 UTC m=+333.942938994" watchObservedRunningTime="2026-09-07 09:04:35.976169478 +0000 UTC m=+333.945340132"
Sep 07 09:04:44.951824 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:44.951788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-m9zcl"
Sep 07 09:04:46.960473 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:46.960443    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-2bllj"
Sep 07 09:04:48.865781 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.865746    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"]
Sep 07 09:04:48.878466 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.878407    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:48.880916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.880889    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"]
Sep 07 09:04:48.881228 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.881214    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8qrnr\""
Sep 07 09:04:48.881341 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.881255    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 07 09:04:48.882691 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.882671    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 07 09:04:48.882818 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.882781    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 07 09:04:48.882818 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.882792    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 07 09:04:48.933806 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.933769    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52e01574-10fe-4d0f-b511-1b3ba9cb7e8b-tls-certs\") pod \"kube-auth-proxy-5b86566b9f-78vr6\" (UID: \"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:48.933966 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.933823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xdf9\" (UniqueName: \"kubernetes.io/projected/52e01574-10fe-4d0f-b511-1b3ba9cb7e8b-kube-api-access-7xdf9\") pod \"kube-auth-proxy-5b86566b9f-78vr6\" (UID: \"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:48.933966 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:48.933895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52e01574-10fe-4d0f-b511-1b3ba9cb7e8b-tmp\") pod \"kube-auth-proxy-5b86566b9f-78vr6\" (UID: \"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:49.035063 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:49.035031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52e01574-10fe-4d0f-b511-1b3ba9cb7e8b-tls-certs\") pod \"kube-auth-proxy-5b86566b9f-78vr6\" (UID: \"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:49.035243 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:49.035080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdf9\" (UniqueName: \"kubernetes.io/projected/52e01574-10fe-4d0f-b511-1b3ba9cb7e8b-kube-api-access-7xdf9\") pod \"kube-auth-proxy-5b86566b9f-78vr6\" (UID: \"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:49.035243 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:49.035127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52e01574-10fe-4d0f-b511-1b3ba9cb7e8b-tmp\") pod \"kube-auth-proxy-5b86566b9f-78vr6\" (UID: \"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:49.037436 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:49.037407    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52e01574-10fe-4d0f-b511-1b3ba9cb7e8b-tmp\") pod \"kube-auth-proxy-5b86566b9f-78vr6\" (UID: \"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:49.037633 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:49.037615    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52e01574-10fe-4d0f-b511-1b3ba9cb7e8b-tls-certs\") pod \"kube-auth-proxy-5b86566b9f-78vr6\" (UID: \"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:49.046563 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:49.046537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdf9\" (UniqueName: \"kubernetes.io/projected/52e01574-10fe-4d0f-b511-1b3ba9cb7e8b-kube-api-access-7xdf9\") pod \"kube-auth-proxy-5b86566b9f-78vr6\" (UID: \"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:49.189464 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:49.189382    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"
Sep 07 09:04:49.464585 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:49.464510    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6"]
Sep 07 09:04:49.467699 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:04:49.467669    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e01574_10fe_4d0f_b511_1b3ba9cb7e8b.slice/crio-75eedef5e1ffc533bb698b6255adb44760f2f5b8b472759b86f7a5702af893b9 WatchSource:0}: Error finding container 75eedef5e1ffc533bb698b6255adb44760f2f5b8b472759b86f7a5702af893b9: Status 404 returned error can't find the container with id 75eedef5e1ffc533bb698b6255adb44760f2f5b8b472759b86f7a5702af893b9
Sep 07 09:04:50.012429 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:50.012391    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6" event={"ID":"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b","Type":"ContainerStarted","Data":"75eedef5e1ffc533bb698b6255adb44760f2f5b8b472759b86f7a5702af893b9"}
Sep 07 09:04:53.025809 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:53.025773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6" event={"ID":"52e01574-10fe-4d0f-b511-1b3ba9cb7e8b","Type":"ContainerStarted","Data":"cb82dd8002a441ff4120eeee4816a445055d4219ff3456668d9834604bc5f871"}
Sep 07 09:04:53.042196 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:04:53.042144    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-78vr6" podStartSLOduration=1.833398576 podStartE2EDuration="5.042129596s" totalImagesPullingTime="3.20873102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:53.040640715 +0000 UTC m=+351.009811345" watchObservedRunningTime="2026-09-07 09:04:53.042129596 +0000 UTC m=+351.011300246"
Sep 07 09:05:08.432823 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.432729    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"]
Sep 07 09:05:08.437380 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.437356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:05:08.441597 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.441563    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8kjfb\""
Sep 07 09:05:08.442846 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.442821    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 07 09:05:08.457261 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.457232    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"]
Sep 07 09:05:08.502233 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.502192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7b195d2-b4fe-42a2-a39c-ac449c8c9af6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7xhnk\" (UID: \"f7b195d2-b4fe-42a2-a39c-ac449c8c9af6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:05:08.502476 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.502323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86bs2\" (UniqueName: \"kubernetes.io/projected/f7b195d2-b4fe-42a2-a39c-ac449c8c9af6-kube-api-access-86bs2\") pod \"llmisvc-controller-manager-777c6bd7d5-7xhnk\" (UID: \"f7b195d2-b4fe-42a2-a39c-ac449c8c9af6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:05:08.603350 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.603308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86bs2\" (UniqueName: \"kubernetes.io/projected/f7b195d2-b4fe-42a2-a39c-ac449c8c9af6-kube-api-access-86bs2\") pod \"llmisvc-controller-manager-777c6bd7d5-7xhnk\" (UID: \"f7b195d2-b4fe-42a2-a39c-ac449c8c9af6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:05:08.603552 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.603370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7b195d2-b4fe-42a2-a39c-ac449c8c9af6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7xhnk\" (UID: \"f7b195d2-b4fe-42a2-a39c-ac449c8c9af6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:05:08.605999 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.605969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7b195d2-b4fe-42a2-a39c-ac449c8c9af6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7xhnk\" (UID: \"f7b195d2-b4fe-42a2-a39c-ac449c8c9af6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:05:08.622003 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.621972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86bs2\" (UniqueName: \"kubernetes.io/projected/f7b195d2-b4fe-42a2-a39c-ac449c8c9af6-kube-api-access-86bs2\") pod \"llmisvc-controller-manager-777c6bd7d5-7xhnk\" (UID: \"f7b195d2-b4fe-42a2-a39c-ac449c8c9af6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:05:08.747058 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:08.747023    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:05:09.016547 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:09.016515    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"]
Sep 07 09:05:09.019056 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:05:09.019028    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7b195d2_b4fe_42a2_a39c_ac449c8c9af6.slice/crio-393779fee088f8f674e333064c87527ab23d443db10a66f359eff0e7b84d5443 WatchSource:0}: Error finding container 393779fee088f8f674e333064c87527ab23d443db10a66f359eff0e7b84d5443: Status 404 returned error can't find the container with id 393779fee088f8f674e333064c87527ab23d443db10a66f359eff0e7b84d5443
Sep 07 09:05:09.091826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:09.091789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk" event={"ID":"f7b195d2-b4fe-42a2-a39c-ac449c8c9af6","Type":"ContainerStarted","Data":"393779fee088f8f674e333064c87527ab23d443db10a66f359eff0e7b84d5443"}
Sep 07 09:05:13.109667 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:13.109627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk" event={"ID":"f7b195d2-b4fe-42a2-a39c-ac449c8c9af6","Type":"ContainerStarted","Data":"711cc8d412a4f10add8b57bc0d43a8c9da3a95900ad97e902d51ebd50d31ab26"}
Sep 07 09:05:13.110067 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:13.109754    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:05:13.140306 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:13.140229    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk" podStartSLOduration=2.069578131 podStartE2EDuration="5.140214924s" totalImagesPullingTime="3.070636793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:05:13.138687878 +0000 UTC m=+371.107858496" watchObservedRunningTime="2026-09-07 09:05:13.140214924 +0000 UTC m=+371.109385529"
Sep 07 09:05:44.116161 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:05:44.116125    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7xhnk"
Sep 07 09:06:52.307193 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.307157    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"]
Sep 07 09:06:52.309435 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.309416    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:52.311961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.311936    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-s4rqb\""
Sep 07 09:06:52.312229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.312212    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 07 09:06:52.312332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.312297    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 07 09:06:52.322482 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.322456    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"]
Sep 07 09:06:52.384555 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.384517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-565cr\" (UniqueName: \"kubernetes.io/projected/57398385-ab4e-4591-a1de-58de3512a69a-kube-api-access-565cr\") pod \"kuadrant-operator-controller-manager-6bd956ccf-fgc4r\" (UID: \"57398385-ab4e-4591-a1de-58de3512a69a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:52.384736 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.384566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57398385-ab4e-4591-a1de-58de3512a69a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-fgc4r\" (UID: \"57398385-ab4e-4591-a1de-58de3512a69a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:52.486055 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.486009    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57398385-ab4e-4591-a1de-58de3512a69a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-fgc4r\" (UID: \"57398385-ab4e-4591-a1de-58de3512a69a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:52.486253 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.486111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-565cr\" (UniqueName: \"kubernetes.io/projected/57398385-ab4e-4591-a1de-58de3512a69a-kube-api-access-565cr\") pod \"kuadrant-operator-controller-manager-6bd956ccf-fgc4r\" (UID: \"57398385-ab4e-4591-a1de-58de3512a69a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:52.486446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.486422    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57398385-ab4e-4591-a1de-58de3512a69a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-fgc4r\" (UID: \"57398385-ab4e-4591-a1de-58de3512a69a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:52.501163 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.501129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-565cr\" (UniqueName: \"kubernetes.io/projected/57398385-ab4e-4591-a1de-58de3512a69a-kube-api-access-565cr\") pod \"kuadrant-operator-controller-manager-6bd956ccf-fgc4r\" (UID: \"57398385-ab4e-4591-a1de-58de3512a69a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:52.619301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.619194    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:52.891679 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.891644    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"]
Sep 07 09:06:52.894574 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:06:52.894548    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57398385_ab4e_4591_a1de_58de3512a69a.slice/crio-79f6d464281111e7f2a0d44870de0e3caea67db2c51aaa4a06a602105adc68a4 WatchSource:0}: Error finding container 79f6d464281111e7f2a0d44870de0e3caea67db2c51aaa4a06a602105adc68a4: Status 404 returned error can't find the container with id 79f6d464281111e7f2a0d44870de0e3caea67db2c51aaa4a06a602105adc68a4
Sep 07 09:06:52.957916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.957882    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"]
Sep 07 09:06:52.970480 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.970448    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"]
Sep 07 09:06:52.984075 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.984044    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"]
Sep 07 09:06:52.986935 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:52.986915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:06:53.002344 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:53.002312    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"]
Sep 07 09:06:53.091925 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:53.091875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whswd\" (UniqueName: \"kubernetes.io/projected/ae4f78f4-1c93-454b-9bad-9179c2682778-kube-api-access-whswd\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jtv9w\" (UID: \"ae4f78f4-1c93-454b-9bad-9179c2682778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:06:53.091925 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:53.091926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae4f78f4-1c93-454b-9bad-9179c2682778-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jtv9w\" (UID: \"ae4f78f4-1c93-454b-9bad-9179c2682778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:06:53.193080 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:53.192985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae4f78f4-1c93-454b-9bad-9179c2682778-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jtv9w\" (UID: \"ae4f78f4-1c93-454b-9bad-9179c2682778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:06:53.193080 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:53.193077    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whswd\" (UniqueName: \"kubernetes.io/projected/ae4f78f4-1c93-454b-9bad-9179c2682778-kube-api-access-whswd\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jtv9w\" (UID: \"ae4f78f4-1c93-454b-9bad-9179c2682778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:06:53.193461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:53.193439    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae4f78f4-1c93-454b-9bad-9179c2682778-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jtv9w\" (UID: \"ae4f78f4-1c93-454b-9bad-9179c2682778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:06:53.203401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:53.203367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whswd\" (UniqueName: \"kubernetes.io/projected/ae4f78f4-1c93-454b-9bad-9179c2682778-kube-api-access-whswd\") pod \"kuadrant-operator-controller-manager-6bd956ccf-jtv9w\" (UID: \"ae4f78f4-1c93-454b-9bad-9179c2682778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:06:53.298807 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:53.298704    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:06:53.614034 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:53.613821    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"]
Sep 07 09:06:53.617183 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:06:53.617137    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae4f78f4_1c93_454b_9bad_9179c2682778.slice/crio-20b2ae3507ed39db6958585569e8aca836086f856f070ecd688543797bf08317 WatchSource:0}: Error finding container 20b2ae3507ed39db6958585569e8aca836086f856f070ecd688543797bf08317: Status 404 returned error can't find the container with id 20b2ae3507ed39db6958585569e8aca836086f856f070ecd688543797bf08317
Sep 07 09:06:54.523878 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:54.523831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w" event={"ID":"ae4f78f4-1c93-454b-9bad-9179c2682778","Type":"ContainerStarted","Data":"20b2ae3507ed39db6958585569e8aca836086f856f070ecd688543797bf08317"}
Sep 07 09:06:57.542835 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.542740    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r" podUID="57398385-ab4e-4591-a1de-58de3512a69a" containerName="manager" containerID="cri-o://dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d" gracePeriod=2
Sep 07 09:06:57.545065 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.545022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w" event={"ID":"ae4f78f4-1c93-454b-9bad-9179c2682778","Type":"ContainerStarted","Data":"301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57"}
Sep 07 09:06:57.545210 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.545177    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:06:57.564335 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.564292    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w" podStartSLOduration=2.541149909 podStartE2EDuration="5.564262801s" totalImagesPullingTime="3.023112892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:06:57.562925148 +0000 UTC m=+475.532095764" watchObservedRunningTime="2026-09-07 09:06:57.564262801 +0000 UTC m=+475.533433407"
Sep 07 09:06:57.779490 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.779466    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:57.781704 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:06:57.781676    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-fgc4r\" is forbidden: User \"system:node:ip-10-0-138-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-157.ec2.internal' and this object" podUID="57398385-ab4e-4591-a1de-58de3512a69a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:57.839461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.839369    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-565cr\" (UniqueName: \"kubernetes.io/projected/57398385-ab4e-4591-a1de-58de3512a69a-kube-api-access-565cr\") pod \"57398385-ab4e-4591-a1de-58de3512a69a\" (UID: \"57398385-ab4e-4591-a1de-58de3512a69a\") "
Sep 07 09:06:57.839461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.839416    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57398385-ab4e-4591-a1de-58de3512a69a-extensions-socket-volume\") pod \"57398385-ab4e-4591-a1de-58de3512a69a\" (UID: \"57398385-ab4e-4591-a1de-58de3512a69a\") "
Sep 07 09:06:57.839785 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.839739    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57398385-ab4e-4591-a1de-58de3512a69a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "57398385-ab4e-4591-a1de-58de3512a69a" (UID: "57398385-ab4e-4591-a1de-58de3512a69a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:06:57.841593 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.841570    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57398385-ab4e-4591-a1de-58de3512a69a-kube-api-access-565cr" (OuterVolumeSpecName: "kube-api-access-565cr") pod "57398385-ab4e-4591-a1de-58de3512a69a" (UID: "57398385-ab4e-4591-a1de-58de3512a69a"). InnerVolumeSpecName "kube-api-access-565cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:06:57.940200 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.940163    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-565cr\" (UniqueName: \"kubernetes.io/projected/57398385-ab4e-4591-a1de-58de3512a69a-kube-api-access-565cr\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:06:57.940200 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:57.940195    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57398385-ab4e-4591-a1de-58de3512a69a-extensions-socket-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:06:58.464450 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:58.464417    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57398385-ab4e-4591-a1de-58de3512a69a" path="/var/lib/kubelet/pods/57398385-ab4e-4591-a1de-58de3512a69a/volumes"
Sep 07 09:06:58.551104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:58.551069    2563 generic.go:356] "Generic (PLEG): container finished" podID="57398385-ab4e-4591-a1de-58de3512a69a" containerID="dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d" exitCode=2
Sep 07 09:06:58.551568 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:58.551117    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:58.551568 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:58.551170    2563 scope.go:117] "RemoveContainer" containerID="dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d"
Sep 07 09:06:58.555735 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:06:58.555700    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-fgc4r\" is forbidden: User \"system:node:ip-10-0-138-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-157.ec2.internal' and this object" podUID="57398385-ab4e-4591-a1de-58de3512a69a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-fgc4r"
Sep 07 09:06:58.559077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:58.559058    2563 scope.go:117] "RemoveContainer" containerID="dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d"
Sep 07 09:06:58.559368 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:06:58.559343    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d\": container with ID starting with dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d not found: ID does not exist" containerID="dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d"
Sep 07 09:06:58.559467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:06:58.559374    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d"} err="failed to get container status \"dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d\": rpc error: code = NotFound desc = could not find container \"dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d\": container with ID starting with dda22052c8ce6234b710e2972bae63ab4f9b98309dda8056d1c92f2b5a4f5a0d not found: ID does not exist"
Sep 07 09:07:08.554461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:08.554427    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:07:20.998121 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:20.998086    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"]
Sep 07 09:07:20.998691 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:20.998365    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w" podUID="ae4f78f4-1c93-454b-9bad-9179c2682778" containerName="manager" containerID="cri-o://301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57" gracePeriod=10
Sep 07 09:07:21.242158 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.242132    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:07:21.340178 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.340091    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae4f78f4-1c93-454b-9bad-9179c2682778-extensions-socket-volume\") pod \"ae4f78f4-1c93-454b-9bad-9179c2682778\" (UID: \"ae4f78f4-1c93-454b-9bad-9179c2682778\") "
Sep 07 09:07:21.340178 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.340129    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whswd\" (UniqueName: \"kubernetes.io/projected/ae4f78f4-1c93-454b-9bad-9179c2682778-kube-api-access-whswd\") pod \"ae4f78f4-1c93-454b-9bad-9179c2682778\" (UID: \"ae4f78f4-1c93-454b-9bad-9179c2682778\") "
Sep 07 09:07:21.340551 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.340520    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae4f78f4-1c93-454b-9bad-9179c2682778-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ae4f78f4-1c93-454b-9bad-9179c2682778" (UID: "ae4f78f4-1c93-454b-9bad-9179c2682778"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:07:21.342229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.342202    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae4f78f4-1c93-454b-9bad-9179c2682778-kube-api-access-whswd" (OuterVolumeSpecName: "kube-api-access-whswd") pod "ae4f78f4-1c93-454b-9bad-9179c2682778" (UID: "ae4f78f4-1c93-454b-9bad-9179c2682778"). InnerVolumeSpecName "kube-api-access-whswd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:07:21.440731 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.440692    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae4f78f4-1c93-454b-9bad-9179c2682778-extensions-socket-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:07:21.440731 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.440723    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whswd\" (UniqueName: \"kubernetes.io/projected/ae4f78f4-1c93-454b-9bad-9179c2682778-kube-api-access-whswd\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:07:21.652875 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.652790    2563 generic.go:356] "Generic (PLEG): container finished" podID="ae4f78f4-1c93-454b-9bad-9179c2682778" containerID="301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57" exitCode=0
Sep 07 09:07:21.652875 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.652834    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"
Sep 07 09:07:21.653062 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.652877    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w" event={"ID":"ae4f78f4-1c93-454b-9bad-9179c2682778","Type":"ContainerDied","Data":"301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57"}
Sep 07 09:07:21.653062 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.652936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w" event={"ID":"ae4f78f4-1c93-454b-9bad-9179c2682778","Type":"ContainerDied","Data":"20b2ae3507ed39db6958585569e8aca836086f856f070ecd688543797bf08317"}
Sep 07 09:07:21.653062 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.652963    2563 scope.go:117] "RemoveContainer" containerID="301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57"
Sep 07 09:07:21.661423 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.661405    2563 scope.go:117] "RemoveContainer" containerID="301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57"
Sep 07 09:07:21.661679 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:07:21.661661    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57\": container with ID starting with 301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57 not found: ID does not exist" containerID="301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57"
Sep 07 09:07:21.661739 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.661692    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57"} err="failed to get container status \"301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57\": rpc error: code = NotFound desc = could not find container \"301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57\": container with ID starting with 301d8a242cfac3430e371507ac917709789af6d92065d1cb9c65dd506c87ae57 not found: ID does not exist"
Sep 07 09:07:21.674526 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.674501    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"]
Sep 07 09:07:21.678536 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:21.678511    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-jtv9w"]
Sep 07 09:07:22.464032 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:22.463997    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae4f78f4-1c93-454b-9bad-9179c2682778" path="/var/lib/kubelet/pods/ae4f78f4-1c93-454b-9bad-9179c2682778/volumes"
Sep 07 09:07:40.091140 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.091106    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"]
Sep 07 09:07:40.091608 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.091479    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57398385-ab4e-4591-a1de-58de3512a69a" containerName="manager"
Sep 07 09:07:40.091608 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.091491    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57398385-ab4e-4591-a1de-58de3512a69a" containerName="manager"
Sep 07 09:07:40.091608 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.091501    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae4f78f4-1c93-454b-9bad-9179c2682778" containerName="manager"
Sep 07 09:07:40.091608 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.091507    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae4f78f4-1c93-454b-9bad-9179c2682778" containerName="manager"
Sep 07 09:07:40.091608 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.091556    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="57398385-ab4e-4591-a1de-58de3512a69a" containerName="manager"
Sep 07 09:07:40.091608 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.091567    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae4f78f4-1c93-454b-9bad-9179c2682778" containerName="manager"
Sep 07 09:07:40.094463 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.094446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:40.097366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.097348    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 07 09:07:40.097570 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.097555    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 07 09:07:40.098569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.098548    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5x592\""
Sep 07 09:07:40.098645 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.098622    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 07 09:07:40.105037 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.105012    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"]
Sep 07 09:07:40.191045 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.191008    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"]
Sep 07 09:07:40.205707 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.205676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a1df696e-dbd5-4e08-9cd0-4840079decda-config-file\") pod \"limitador-limitador-859c4d96d6-tnzx8\" (UID: \"a1df696e-dbd5-4e08-9cd0-4840079decda\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:40.205854 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.205747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zlp8\" (UniqueName: \"kubernetes.io/projected/a1df696e-dbd5-4e08-9cd0-4840079decda-kube-api-access-4zlp8\") pod \"limitador-limitador-859c4d96d6-tnzx8\" (UID: \"a1df696e-dbd5-4e08-9cd0-4840079decda\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:40.307021 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.306966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zlp8\" (UniqueName: \"kubernetes.io/projected/a1df696e-dbd5-4e08-9cd0-4840079decda-kube-api-access-4zlp8\") pod \"limitador-limitador-859c4d96d6-tnzx8\" (UID: \"a1df696e-dbd5-4e08-9cd0-4840079decda\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:40.307202 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.307059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a1df696e-dbd5-4e08-9cd0-4840079decda-config-file\") pod \"limitador-limitador-859c4d96d6-tnzx8\" (UID: \"a1df696e-dbd5-4e08-9cd0-4840079decda\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:40.307689 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.307670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a1df696e-dbd5-4e08-9cd0-4840079decda-config-file\") pod \"limitador-limitador-859c4d96d6-tnzx8\" (UID: \"a1df696e-dbd5-4e08-9cd0-4840079decda\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:40.314701 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.314672    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zlp8\" (UniqueName: \"kubernetes.io/projected/a1df696e-dbd5-4e08-9cd0-4840079decda-kube-api-access-4zlp8\") pod \"limitador-limitador-859c4d96d6-tnzx8\" (UID: \"a1df696e-dbd5-4e08-9cd0-4840079decda\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:40.404711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.404621    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:40.673960 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.673935    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"]
Sep 07 09:07:40.676131 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:07:40.676100    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1df696e_dbd5_4e08_9cd0_4840079decda.slice/crio-05123690d2bc27fb9b0b96d57874e066042c3544e1612a198714eaa8b881e85e WatchSource:0}: Error finding container 05123690d2bc27fb9b0b96d57874e066042c3544e1612a198714eaa8b881e85e: Status 404 returned error can't find the container with id 05123690d2bc27fb9b0b96d57874e066042c3544e1612a198714eaa8b881e85e
Sep 07 09:07:40.743423 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.743388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8" event={"ID":"a1df696e-dbd5-4e08-9cd0-4840079decda","Type":"ContainerStarted","Data":"05123690d2bc27fb9b0b96d57874e066042c3544e1612a198714eaa8b881e85e"}
Sep 07 09:07:40.885149 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.885116    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-sb4bb"]
Sep 07 09:07:40.889569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.889542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-sb4bb"
Sep 07 09:07:40.891903 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.891868    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hp9wv\""
Sep 07 09:07:40.895071 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:40.895043    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-sb4bb"]
Sep 07 09:07:41.012395 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.012358    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47t6v\" (UniqueName: \"kubernetes.io/projected/7b67afaf-a470-47fa-8201-e31eb118b7c0-kube-api-access-47t6v\") pod \"authorino-b54df94b4-sb4bb\" (UID: \"7b67afaf-a470-47fa-8201-e31eb118b7c0\") " pod="kuadrant-system/authorino-b54df94b4-sb4bb"
Sep 07 09:07:41.084507 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.084467    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-kzjkq"]
Sep 07 09:07:41.087810 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.087792    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-kzjkq"
Sep 07 09:07:41.091651 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.091624    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-kzjkq"]
Sep 07 09:07:41.113237 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.113200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47t6v\" (UniqueName: \"kubernetes.io/projected/7b67afaf-a470-47fa-8201-e31eb118b7c0-kube-api-access-47t6v\") pod \"authorino-b54df94b4-sb4bb\" (UID: \"7b67afaf-a470-47fa-8201-e31eb118b7c0\") " pod="kuadrant-system/authorino-b54df94b4-sb4bb"
Sep 07 09:07:41.122044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.122017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47t6v\" (UniqueName: \"kubernetes.io/projected/7b67afaf-a470-47fa-8201-e31eb118b7c0-kube-api-access-47t6v\") pod \"authorino-b54df94b4-sb4bb\" (UID: \"7b67afaf-a470-47fa-8201-e31eb118b7c0\") " pod="kuadrant-system/authorino-b54df94b4-sb4bb"
Sep 07 09:07:41.214347 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.214308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvmtr\" (UniqueName: \"kubernetes.io/projected/a9fd5b4a-7a85-441d-8d96-8fd9391359b1-kube-api-access-zvmtr\") pod \"authorino-c548b5d7d-kzjkq\" (UID: \"a9fd5b4a-7a85-441d-8d96-8fd9391359b1\") " pod="kuadrant-system/authorino-c548b5d7d-kzjkq"
Sep 07 09:07:41.214515 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.214427    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-sb4bb"
Sep 07 09:07:41.315182 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.315143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvmtr\" (UniqueName: \"kubernetes.io/projected/a9fd5b4a-7a85-441d-8d96-8fd9391359b1-kube-api-access-zvmtr\") pod \"authorino-c548b5d7d-kzjkq\" (UID: \"a9fd5b4a-7a85-441d-8d96-8fd9391359b1\") " pod="kuadrant-system/authorino-c548b5d7d-kzjkq"
Sep 07 09:07:41.327920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.327891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvmtr\" (UniqueName: \"kubernetes.io/projected/a9fd5b4a-7a85-441d-8d96-8fd9391359b1-kube-api-access-zvmtr\") pod \"authorino-c548b5d7d-kzjkq\" (UID: \"a9fd5b4a-7a85-441d-8d96-8fd9391359b1\") " pod="kuadrant-system/authorino-c548b5d7d-kzjkq"
Sep 07 09:07:41.397806 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.397771    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-kzjkq"
Sep 07 09:07:41.485297 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.485248    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-sb4bb"]
Sep 07 09:07:41.488690 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:07:41.488647    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b67afaf_a470_47fa_8201_e31eb118b7c0.slice/crio-920f11416ff1eefa1bab52d3ea8334c4a9fb2bfd935171885c4f91cd618477d9 WatchSource:0}: Error finding container 920f11416ff1eefa1bab52d3ea8334c4a9fb2bfd935171885c4f91cd618477d9: Status 404 returned error can't find the container with id 920f11416ff1eefa1bab52d3ea8334c4a9fb2bfd935171885c4f91cd618477d9
Sep 07 09:07:41.680004 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.679950    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-kzjkq"]
Sep 07 09:07:41.749110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:41.749080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-sb4bb" event={"ID":"7b67afaf-a470-47fa-8201-e31eb118b7c0","Type":"ContainerStarted","Data":"920f11416ff1eefa1bab52d3ea8334c4a9fb2bfd935171885c4f91cd618477d9"}
Sep 07 09:07:41.757872 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:07:41.757846    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9fd5b4a_7a85_441d_8d96_8fd9391359b1.slice/crio-00da703654ccc00005012d2c66b38ca4f01fa6e48607192be7deac7c6199f516 WatchSource:0}: Error finding container 00da703654ccc00005012d2c66b38ca4f01fa6e48607192be7deac7c6199f516: Status 404 returned error can't find the container with id 00da703654ccc00005012d2c66b38ca4f01fa6e48607192be7deac7c6199f516
Sep 07 09:07:42.759631 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:42.758742    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8" event={"ID":"a1df696e-dbd5-4e08-9cd0-4840079decda","Type":"ContainerStarted","Data":"548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4"}
Sep 07 09:07:42.759631 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:42.759235    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:42.761524 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:42.761454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-kzjkq" event={"ID":"a9fd5b4a-7a85-441d-8d96-8fd9391359b1","Type":"ContainerStarted","Data":"00da703654ccc00005012d2c66b38ca4f01fa6e48607192be7deac7c6199f516"}
Sep 07 09:07:42.775921 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:42.775006    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8" podStartSLOduration=1.641139919 podStartE2EDuration="2.774984924s" totalImagesPullingTime="1.133845005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:07:42.773428696 +0000 UTC m=+520.742599302" watchObservedRunningTime="2026-09-07 09:07:42.774984924 +0000 UTC m=+520.744155530"
Sep 07 09:07:44.772669 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:44.772626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-sb4bb" event={"ID":"7b67afaf-a470-47fa-8201-e31eb118b7c0","Type":"ContainerStarted","Data":"fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1"}
Sep 07 09:07:44.774227 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:44.774201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-kzjkq" event={"ID":"a9fd5b4a-7a85-441d-8d96-8fd9391359b1","Type":"ContainerStarted","Data":"8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf"}
Sep 07 09:07:44.786773 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:44.786713    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-sb4bb" podStartSLOduration=2.39569973 podStartE2EDuration="4.786698008s" totalImagesPullingTime="2.390998278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:07:44.785813456 +0000 UTC m=+522.754984064" watchObservedRunningTime="2026-09-07 09:07:44.786698008 +0000 UTC m=+522.755868613"
Sep 07 09:07:44.800834 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:44.800779    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-kzjkq" podStartSLOduration=1.691447066 podStartE2EDuration="3.80076012s" totalImagesPullingTime="2.109313054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:07:44.799977899 +0000 UTC m=+522.769148505" watchObservedRunningTime="2026-09-07 09:07:44.80076012 +0000 UTC m=+522.769930724"
Sep 07 09:07:44.825400 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:44.825362    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-sb4bb"]
Sep 07 09:07:46.783543 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:46.783507    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-sb4bb" podUID="7b67afaf-a470-47fa-8201-e31eb118b7c0" containerName="authorino" containerID="cri-o://fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1" gracePeriod=30
Sep 07 09:07:47.030749 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.030722    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-sb4bb"
Sep 07 09:07:47.171898 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.171803    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47t6v\" (UniqueName: \"kubernetes.io/projected/7b67afaf-a470-47fa-8201-e31eb118b7c0-kube-api-access-47t6v\") pod \"7b67afaf-a470-47fa-8201-e31eb118b7c0\" (UID: \"7b67afaf-a470-47fa-8201-e31eb118b7c0\") "
Sep 07 09:07:47.173952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.173920    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b67afaf-a470-47fa-8201-e31eb118b7c0-kube-api-access-47t6v" (OuterVolumeSpecName: "kube-api-access-47t6v") pod "7b67afaf-a470-47fa-8201-e31eb118b7c0" (UID: "7b67afaf-a470-47fa-8201-e31eb118b7c0"). InnerVolumeSpecName "kube-api-access-47t6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:07:47.272678 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.272631    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47t6v\" (UniqueName: \"kubernetes.io/projected/7b67afaf-a470-47fa-8201-e31eb118b7c0-kube-api-access-47t6v\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:07:47.789483 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.789448    2563 generic.go:356] "Generic (PLEG): container finished" podID="7b67afaf-a470-47fa-8201-e31eb118b7c0" containerID="fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1" exitCode=0
Sep 07 09:07:47.789955 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.789492    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-sb4bb"
Sep 07 09:07:47.789955 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.789532    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-sb4bb" event={"ID":"7b67afaf-a470-47fa-8201-e31eb118b7c0","Type":"ContainerDied","Data":"fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1"}
Sep 07 09:07:47.789955 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.789576    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-sb4bb" event={"ID":"7b67afaf-a470-47fa-8201-e31eb118b7c0","Type":"ContainerDied","Data":"920f11416ff1eefa1bab52d3ea8334c4a9fb2bfd935171885c4f91cd618477d9"}
Sep 07 09:07:47.789955 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.789596    2563 scope.go:117] "RemoveContainer" containerID="fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1"
Sep 07 09:07:47.798224 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.798207    2563 scope.go:117] "RemoveContainer" containerID="fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1"
Sep 07 09:07:47.798497 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:07:47.798477    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1\": container with ID starting with fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1 not found: ID does not exist" containerID="fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1"
Sep 07 09:07:47.798560 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.798507    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1"} err="failed to get container status \"fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1\": rpc error: code = NotFound desc = could not find container \"fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1\": container with ID starting with fc062341b88c57cb04ed34a6545a1b03f6c1cc5cdcb519b72d7b52e7614877a1 not found: ID does not exist"
Sep 07 09:07:47.809025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.808993    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-sb4bb"]
Sep 07 09:07:47.815629 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:47.812730    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-sb4bb"]
Sep 07 09:07:48.464480 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:48.464445    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b67afaf-a470-47fa-8201-e31eb118b7c0" path="/var/lib/kubelet/pods/7b67afaf-a470-47fa-8201-e31eb118b7c0/volumes"
Sep 07 09:07:53.149755 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.149722    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"]
Sep 07 09:07:53.150244 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.149973    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8" podUID="a1df696e-dbd5-4e08-9cd0-4840079decda" containerName="limitador" containerID="cri-o://548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4" gracePeriod=30
Sep 07 09:07:53.150693 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.150676    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:53.689798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.689768    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:53.818888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.818854    2563 generic.go:356] "Generic (PLEG): container finished" podID="a1df696e-dbd5-4e08-9cd0-4840079decda" containerID="548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4" exitCode=0
Sep 07 09:07:53.819039 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.818897    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"
Sep 07 09:07:53.819039 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.818909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8" event={"ID":"a1df696e-dbd5-4e08-9cd0-4840079decda","Type":"ContainerDied","Data":"548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4"}
Sep 07 09:07:53.819039 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.818957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tnzx8" event={"ID":"a1df696e-dbd5-4e08-9cd0-4840079decda","Type":"ContainerDied","Data":"05123690d2bc27fb9b0b96d57874e066042c3544e1612a198714eaa8b881e85e"}
Sep 07 09:07:53.819039 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.818981    2563 scope.go:117] "RemoveContainer" containerID="548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4"
Sep 07 09:07:53.827276 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.827255    2563 scope.go:117] "RemoveContainer" containerID="548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4"
Sep 07 09:07:53.827576 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:07:53.827556    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4\": container with ID starting with 548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4 not found: ID does not exist" containerID="548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4"
Sep 07 09:07:53.827635 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.827584    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4"} err="failed to get container status \"548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4\": rpc error: code = NotFound desc = could not find container \"548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4\": container with ID starting with 548535a3e0f7aa282f47d3304142d47e6a9e4a80a92721d4932808d8271a84a4 not found: ID does not exist"
Sep 07 09:07:53.833889 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.833866    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zlp8\" (UniqueName: \"kubernetes.io/projected/a1df696e-dbd5-4e08-9cd0-4840079decda-kube-api-access-4zlp8\") pod \"a1df696e-dbd5-4e08-9cd0-4840079decda\" (UID: \"a1df696e-dbd5-4e08-9cd0-4840079decda\") "
Sep 07 09:07:53.833985 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.833962    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a1df696e-dbd5-4e08-9cd0-4840079decda-config-file\") pod \"a1df696e-dbd5-4e08-9cd0-4840079decda\" (UID: \"a1df696e-dbd5-4e08-9cd0-4840079decda\") "
Sep 07 09:07:53.834262 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.834242    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1df696e-dbd5-4e08-9cd0-4840079decda-config-file" (OuterVolumeSpecName: "config-file") pod "a1df696e-dbd5-4e08-9cd0-4840079decda" (UID: "a1df696e-dbd5-4e08-9cd0-4840079decda"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:07:53.835831 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.835808    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1df696e-dbd5-4e08-9cd0-4840079decda-kube-api-access-4zlp8" (OuterVolumeSpecName: "kube-api-access-4zlp8") pod "a1df696e-dbd5-4e08-9cd0-4840079decda" (UID: "a1df696e-dbd5-4e08-9cd0-4840079decda"). InnerVolumeSpecName "kube-api-access-4zlp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:07:53.934783 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.934748    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a1df696e-dbd5-4e08-9cd0-4840079decda-config-file\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:07:53.934783 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:53.934778    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zlp8\" (UniqueName: \"kubernetes.io/projected/a1df696e-dbd5-4e08-9cd0-4840079decda-kube-api-access-4zlp8\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:07:54.145253 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:54.145221    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"]
Sep 07 09:07:54.148798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:54.148775    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tnzx8"]
Sep 07 09:07:54.464798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:54.464699    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1df696e-dbd5-4e08-9cd0-4840079decda" path="/var/lib/kubelet/pods/a1df696e-dbd5-4e08-9cd0-4840079decda/volumes"
Sep 07 09:07:59.527881 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.527793    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"]
Sep 07 09:07:59.528343 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.528325    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b67afaf-a470-47fa-8201-e31eb118b7c0" containerName="authorino"
Sep 07 09:07:59.528404 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.528346    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b67afaf-a470-47fa-8201-e31eb118b7c0" containerName="authorino"
Sep 07 09:07:59.528404 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.528386    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1df696e-dbd5-4e08-9cd0-4840079decda" containerName="limitador"
Sep 07 09:07:59.528404 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.528396    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1df696e-dbd5-4e08-9cd0-4840079decda" containerName="limitador"
Sep 07 09:07:59.528496 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.528465    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b67afaf-a470-47fa-8201-e31eb118b7c0" containerName="authorino"
Sep 07 09:07:59.528496 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.528480    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1df696e-dbd5-4e08-9cd0-4840079decda" containerName="limitador"
Sep 07 09:07:59.531418 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.531390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:07:59.534361 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.534334    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 09:07:59.534361 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.534352    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 09:07:59.534544 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.534342    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 07 09:07:59.535373 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.535354    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-w48tl\""
Sep 07 09:07:59.539603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.539581    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"]
Sep 07 09:07:59.681620 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.681575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcb2p\" (UniqueName: \"kubernetes.io/projected/112badc2-95a3-448b-b7bd-1897bf55dbd2-kube-api-access-rcb2p\") pod \"postgres-69fb4594cf-rgtqx\" (UID: \"112badc2-95a3-448b-b7bd-1897bf55dbd2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:07:59.681787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.681629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/112badc2-95a3-448b-b7bd-1897bf55dbd2-data\") pod \"postgres-69fb4594cf-rgtqx\" (UID: \"112badc2-95a3-448b-b7bd-1897bf55dbd2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:07:59.782534 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.782433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcb2p\" (UniqueName: \"kubernetes.io/projected/112badc2-95a3-448b-b7bd-1897bf55dbd2-kube-api-access-rcb2p\") pod \"postgres-69fb4594cf-rgtqx\" (UID: \"112badc2-95a3-448b-b7bd-1897bf55dbd2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:07:59.782534 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.782497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/112badc2-95a3-448b-b7bd-1897bf55dbd2-data\") pod \"postgres-69fb4594cf-rgtqx\" (UID: \"112badc2-95a3-448b-b7bd-1897bf55dbd2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:07:59.782878 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.782858    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/112badc2-95a3-448b-b7bd-1897bf55dbd2-data\") pod \"postgres-69fb4594cf-rgtqx\" (UID: \"112badc2-95a3-448b-b7bd-1897bf55dbd2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:07:59.792115 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.792083    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcb2p\" (UniqueName: \"kubernetes.io/projected/112badc2-95a3-448b-b7bd-1897bf55dbd2-kube-api-access-rcb2p\") pod \"postgres-69fb4594cf-rgtqx\" (UID: \"112badc2-95a3-448b-b7bd-1897bf55dbd2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:07:59.843227 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:07:59.843196    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:08:00.111880 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:00.111854    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"]
Sep 07 09:08:00.114190 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:08:00.114164    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod112badc2_95a3_448b_b7bd_1897bf55dbd2.slice/crio-a86058141c6a49717390c3f91d8b1022ca98bfdaf816df6c671f60c6c4890259 WatchSource:0}: Error finding container a86058141c6a49717390c3f91d8b1022ca98bfdaf816df6c671f60c6c4890259: Status 404 returned error can't find the container with id a86058141c6a49717390c3f91d8b1022ca98bfdaf816df6c671f60c6c4890259
Sep 07 09:08:00.859605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:00.859550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx" event={"ID":"112badc2-95a3-448b-b7bd-1897bf55dbd2","Type":"ContainerStarted","Data":"a86058141c6a49717390c3f91d8b1022ca98bfdaf816df6c671f60c6c4890259"}
Sep 07 09:08:04.855541 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:04.855515    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 07 09:08:05.891562 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:05.891519    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx" event={"ID":"112badc2-95a3-448b-b7bd-1897bf55dbd2","Type":"ContainerStarted","Data":"0bc11415ed74abec308251efaf9822e1916a597e49764ee1758f354b0bb27149"}
Sep 07 09:08:05.892070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:05.891616    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:08:05.907694 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:05.907646    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx" podStartSLOduration=2.170441625 podStartE2EDuration="6.907630479s" totalImagesPullingTime="4.737188854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:08:05.906305258 +0000 UTC m=+543.875475863" watchObservedRunningTime="2026-09-07 09:08:05.907630479 +0000 UTC m=+543.876801083"
Sep 07 09:08:11.925695 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:11.925667    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rgtqx"
Sep 07 09:08:12.445760 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.445719    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xtt9f"]
Sep 07 09:08:12.450729 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.450709    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xtt9f"
Sep 07 09:08:12.455835 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.455803    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xtt9f"]
Sep 07 09:08:12.471706 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.471666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77hl6\" (UniqueName: \"kubernetes.io/projected/7b051b45-6fc4-4ee7-8998-8ede156e32ac-kube-api-access-77hl6\") pod \"authorino-5dc7ccb5f-xtt9f\" (UID: \"7b051b45-6fc4-4ee7-8998-8ede156e32ac\") " pod="kuadrant-system/authorino-5dc7ccb5f-xtt9f"
Sep 07 09:08:12.527379 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.527342    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xtt9f"]
Sep 07 09:08:12.527651 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:08:12.527629    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-77hl6], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-xtt9f" podUID="7b051b45-6fc4-4ee7-8998-8ede156e32ac"
Sep 07 09:08:12.572454 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.572421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77hl6\" (UniqueName: \"kubernetes.io/projected/7b051b45-6fc4-4ee7-8998-8ede156e32ac-kube-api-access-77hl6\") pod \"authorino-5dc7ccb5f-xtt9f\" (UID: \"7b051b45-6fc4-4ee7-8998-8ede156e32ac\") " pod="kuadrant-system/authorino-5dc7ccb5f-xtt9f"
Sep 07 09:08:12.580271 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.580235    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77hl6\" (UniqueName: \"kubernetes.io/projected/7b051b45-6fc4-4ee7-8998-8ede156e32ac-kube-api-access-77hl6\") pod \"authorino-5dc7ccb5f-xtt9f\" (UID: \"7b051b45-6fc4-4ee7-8998-8ede156e32ac\") " pod="kuadrant-system/authorino-5dc7ccb5f-xtt9f"
Sep 07 09:08:12.712594 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.712505    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7989bc5db8-pj9g2"]
Sep 07 09:08:12.717720 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.717701    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:08:12.720419 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.720392    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 07 09:08:12.721824 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.721800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7989bc5db8-pj9g2"]
Sep 07 09:08:12.774053 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.773983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac5f019-c809-4793-9214-1033bc8c0a7b-tls-cert\") pod \"authorino-7989bc5db8-pj9g2\" (UID: \"5ac5f019-c809-4793-9214-1033bc8c0a7b\") " pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:08:12.774232 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.774108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtg6d\" (UniqueName: \"kubernetes.io/projected/5ac5f019-c809-4793-9214-1033bc8c0a7b-kube-api-access-xtg6d\") pod \"authorino-7989bc5db8-pj9g2\" (UID: \"5ac5f019-c809-4793-9214-1033bc8c0a7b\") " pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:08:12.874965 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.874921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac5f019-c809-4793-9214-1033bc8c0a7b-tls-cert\") pod \"authorino-7989bc5db8-pj9g2\" (UID: \"5ac5f019-c809-4793-9214-1033bc8c0a7b\") " pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:08:12.875138 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.874985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtg6d\" (UniqueName: \"kubernetes.io/projected/5ac5f019-c809-4793-9214-1033bc8c0a7b-kube-api-access-xtg6d\") pod \"authorino-7989bc5db8-pj9g2\" (UID: \"5ac5f019-c809-4793-9214-1033bc8c0a7b\") " pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:08:12.877447 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.877421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac5f019-c809-4793-9214-1033bc8c0a7b-tls-cert\") pod \"authorino-7989bc5db8-pj9g2\" (UID: \"5ac5f019-c809-4793-9214-1033bc8c0a7b\") " pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:08:12.883250 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.883215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtg6d\" (UniqueName: \"kubernetes.io/projected/5ac5f019-c809-4793-9214-1033bc8c0a7b-kube-api-access-xtg6d\") pod \"authorino-7989bc5db8-pj9g2\" (UID: \"5ac5f019-c809-4793-9214-1033bc8c0a7b\") " pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:08:12.922372 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.922329    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xtt9f"
Sep 07 09:08:12.936018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.935929    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xtt9f"
Sep 07 09:08:12.975804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.975713    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77hl6\" (UniqueName: \"kubernetes.io/projected/7b051b45-6fc4-4ee7-8998-8ede156e32ac-kube-api-access-77hl6\") pod \"7b051b45-6fc4-4ee7-8998-8ede156e32ac\" (UID: \"7b051b45-6fc4-4ee7-8998-8ede156e32ac\") "
Sep 07 09:08:12.977817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:12.977782    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b051b45-6fc4-4ee7-8998-8ede156e32ac-kube-api-access-77hl6" (OuterVolumeSpecName: "kube-api-access-77hl6") pod "7b051b45-6fc4-4ee7-8998-8ede156e32ac" (UID: "7b051b45-6fc4-4ee7-8998-8ede156e32ac"). InnerVolumeSpecName "kube-api-access-77hl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:08:13.027687 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:13.027648    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:08:13.076785 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:13.076740    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77hl6\" (UniqueName: \"kubernetes.io/projected/7b051b45-6fc4-4ee7-8998-8ede156e32ac-kube-api-access-77hl6\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:08:13.302424 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:13.302398    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7989bc5db8-pj9g2"]
Sep 07 09:08:13.304312 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:08:13.304273    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ac5f019_c809_4793_9214_1033bc8c0a7b.slice/crio-f0237b5f8ac676c883d256a6c87a456b1cc254f223af7112851df3a8528ea83c WatchSource:0}: Error finding container f0237b5f8ac676c883d256a6c87a456b1cc254f223af7112851df3a8528ea83c: Status 404 returned error can't find the container with id f0237b5f8ac676c883d256a6c87a456b1cc254f223af7112851df3a8528ea83c
Sep 07 09:08:13.928300 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:13.928255    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xtt9f"
Sep 07 09:08:13.928480 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:13.928262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7989bc5db8-pj9g2" event={"ID":"5ac5f019-c809-4793-9214-1033bc8c0a7b","Type":"ContainerStarted","Data":"f0237b5f8ac676c883d256a6c87a456b1cc254f223af7112851df3a8528ea83c"}
Sep 07 09:08:13.956604 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:13.956573    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xtt9f"]
Sep 07 09:08:13.960392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:13.960364    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xtt9f"]
Sep 07 09:08:14.463989 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:14.463946    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b051b45-6fc4-4ee7-8998-8ede156e32ac" path="/var/lib/kubelet/pods/7b051b45-6fc4-4ee7-8998-8ede156e32ac/volumes"
Sep 07 09:08:16.942688 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:16.942653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7989bc5db8-pj9g2" event={"ID":"5ac5f019-c809-4793-9214-1033bc8c0a7b","Type":"ContainerStarted","Data":"41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3"}
Sep 07 09:08:16.958506 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:16.958451    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7989bc5db8-pj9g2" podStartSLOduration=2.271391947 podStartE2EDuration="4.958431913s" totalImagesPullingTime="2.687039966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:08:16.957663467 +0000 UTC m=+554.926834074" watchObservedRunningTime="2026-09-07 09:08:16.958431913 +0000 UTC m=+554.927602518"
Sep 07 09:08:16.980707 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:16.980671    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-kzjkq"]
Sep 07 09:08:16.980926 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:16.980901    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-kzjkq" podUID="a9fd5b4a-7a85-441d-8d96-8fd9391359b1" containerName="authorino" containerID="cri-o://8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf" gracePeriod=30
Sep 07 09:08:17.227392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.227363    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-kzjkq"
Sep 07 09:08:17.307026 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.306984    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvmtr\" (UniqueName: \"kubernetes.io/projected/a9fd5b4a-7a85-441d-8d96-8fd9391359b1-kube-api-access-zvmtr\") pod \"a9fd5b4a-7a85-441d-8d96-8fd9391359b1\" (UID: \"a9fd5b4a-7a85-441d-8d96-8fd9391359b1\") "
Sep 07 09:08:17.309109 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.309072    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9fd5b4a-7a85-441d-8d96-8fd9391359b1-kube-api-access-zvmtr" (OuterVolumeSpecName: "kube-api-access-zvmtr") pod "a9fd5b4a-7a85-441d-8d96-8fd9391359b1" (UID: "a9fd5b4a-7a85-441d-8d96-8fd9391359b1"). InnerVolumeSpecName "kube-api-access-zvmtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:08:17.407841 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.407797    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvmtr\" (UniqueName: \"kubernetes.io/projected/a9fd5b4a-7a85-441d-8d96-8fd9391359b1-kube-api-access-zvmtr\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:08:17.947689 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.947651    2563 generic.go:356] "Generic (PLEG): container finished" podID="a9fd5b4a-7a85-441d-8d96-8fd9391359b1" containerID="8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf" exitCode=0
Sep 07 09:08:17.948076 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.947711    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-kzjkq"
Sep 07 09:08:17.948076 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.947727    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-kzjkq" event={"ID":"a9fd5b4a-7a85-441d-8d96-8fd9391359b1","Type":"ContainerDied","Data":"8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf"}
Sep 07 09:08:17.948076 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.947769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-kzjkq" event={"ID":"a9fd5b4a-7a85-441d-8d96-8fd9391359b1","Type":"ContainerDied","Data":"00da703654ccc00005012d2c66b38ca4f01fa6e48607192be7deac7c6199f516"}
Sep 07 09:08:17.948076 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.947787    2563 scope.go:117] "RemoveContainer" containerID="8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf"
Sep 07 09:08:17.956633 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.956614    2563 scope.go:117] "RemoveContainer" containerID="8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf"
Sep 07 09:08:17.956917 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:08:17.956898    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf\": container with ID starting with 8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf not found: ID does not exist" containerID="8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf"
Sep 07 09:08:17.956961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.956925    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf"} err="failed to get container status \"8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf\": rpc error: code = NotFound desc = could not find container \"8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf\": container with ID starting with 8bb2c253ba0babc4eada908b043c4cd840ec8ac0947efb2a65e5a9a10a0c4daf not found: ID does not exist"
Sep 07 09:08:17.970489 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.970460    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-kzjkq"]
Sep 07 09:08:17.972603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:17.972582    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-kzjkq"]
Sep 07 09:08:18.464354 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:18.464309    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9fd5b4a-7a85-441d-8d96-8fd9391359b1" path="/var/lib/kubelet/pods/a9fd5b4a-7a85-441d-8d96-8fd9391359b1/volumes"
Sep 07 09:08:21.678574 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.678537    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67fc66649-hs5vz"]
Sep 07 09:08:21.678990 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.678909    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9fd5b4a-7a85-441d-8d96-8fd9391359b1" containerName="authorino"
Sep 07 09:08:21.678990 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.678922    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9fd5b4a-7a85-441d-8d96-8fd9391359b1" containerName="authorino"
Sep 07 09:08:21.678990 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.678982    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9fd5b4a-7a85-441d-8d96-8fd9391359b1" containerName="authorino"
Sep 07 09:08:21.681822 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.681801    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:21.684377 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.684350    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 09:08:21.684377 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.684350    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 09:08:21.685361 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.685339    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 09:08:21.685415 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.685372    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mk7x4\""
Sep 07 09:08:21.693203 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.693176    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67fc66649-hs5vz"]
Sep 07 09:08:21.737148 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.737104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-metrics-certs\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:21.737336 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.737162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq6mj\" (UniqueName: \"kubernetes.io/projected/e7cc68bb-cf9c-468b-8658-186f8857d6a9-kube-api-access-hq6mj\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:21.737336 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.737236    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-webhook-cert\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:21.837044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.837004    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6957b5f6b-whncm"]
Sep 07 09:08:21.837737 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.837713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-metrics-certs\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:21.837803 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.837762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq6mj\" (UniqueName: \"kubernetes.io/projected/e7cc68bb-cf9c-468b-8658-186f8857d6a9-kube-api-access-hq6mj\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:21.837803 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.837799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-webhook-cert\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:21.837883 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:08:21.837862    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 07 09:08:21.837915 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:08:21.837889    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 07 09:08:21.837948 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:08:21.837935    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-metrics-certs podName:e7cc68bb-cf9c-468b-8658-186f8857d6a9 nodeName:}" failed. No retries permitted until 2026-09-07 09:08:22.33791944 +0000 UTC m=+560.307090028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-metrics-certs") pod "maas-controller-67fc66649-hs5vz" (UID: "e7cc68bb-cf9c-468b-8658-186f8857d6a9") : secret "maas-controller-metrics-tls" not found
Sep 07 09:08:21.837993 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:08:21.837949    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-webhook-cert podName:e7cc68bb-cf9c-468b-8658-186f8857d6a9 nodeName:}" failed. No retries permitted until 2026-09-07 09:08:22.337943492 +0000 UTC m=+560.307114075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-webhook-cert") pod "maas-controller-67fc66649-hs5vz" (UID: "e7cc68bb-cf9c-468b-8658-186f8857d6a9") : secret "maas-controller-webhook-cert" not found
Sep 07 09:08:21.839354 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.839340    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:21.845999 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.845970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq6mj\" (UniqueName: \"kubernetes.io/projected/e7cc68bb-cf9c-468b-8658-186f8857d6a9-kube-api-access-hq6mj\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:21.851170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.851146    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-whncm"]
Sep 07 09:08:21.938257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.938150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-webhook-cert\") pod \"maas-controller-6957b5f6b-whncm\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") " pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:21.938257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.938202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjsst\" (UniqueName: \"kubernetes.io/projected/32a66938-309e-44da-8c4a-07e1005cf4d6-kube-api-access-rjsst\") pod \"maas-controller-6957b5f6b-whncm\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") " pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:21.938502 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:21.938373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-metrics-certs\") pod \"maas-controller-6957b5f6b-whncm\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") " pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:22.039455 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.039420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-metrics-certs\") pod \"maas-controller-6957b5f6b-whncm\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") " pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:22.039650 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.039595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-webhook-cert\") pod \"maas-controller-6957b5f6b-whncm\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") " pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:22.039650 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.039643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjsst\" (UniqueName: \"kubernetes.io/projected/32a66938-309e-44da-8c4a-07e1005cf4d6-kube-api-access-rjsst\") pod \"maas-controller-6957b5f6b-whncm\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") " pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:22.041841 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.041811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-webhook-cert\") pod \"maas-controller-6957b5f6b-whncm\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") " pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:22.041942 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.041840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-metrics-certs\") pod \"maas-controller-6957b5f6b-whncm\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") " pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:22.047840 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.047812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjsst\" (UniqueName: \"kubernetes.io/projected/32a66938-309e-44da-8c4a-07e1005cf4d6-kube-api-access-rjsst\") pod \"maas-controller-6957b5f6b-whncm\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") " pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:22.157737 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.157700    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:22.342454 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.342407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-webhook-cert\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:22.342650 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.342505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-metrics-certs\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:22.344827 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.344800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-webhook-cert\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:22.344924 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.344897    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-metrics-certs\") pod \"maas-controller-67fc66649-hs5vz\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") " pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:22.425941 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.425838    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-whncm"]
Sep 07 09:08:22.428600 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:08:22.428575    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a66938_309e_44da_8c4a_07e1005cf4d6.slice/crio-ed1c44ef5675b278f49bbab9d212ec4436c4b69bf439b498cf76c6a1be6b8cb0 WatchSource:0}: Error finding container ed1c44ef5675b278f49bbab9d212ec4436c4b69bf439b498cf76c6a1be6b8cb0: Status 404 returned error can't find the container with id ed1c44ef5675b278f49bbab9d212ec4436c4b69bf439b498cf76c6a1be6b8cb0
Sep 07 09:08:22.592296 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.592261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:22.860179 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.860155    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67fc66649-hs5vz"]
Sep 07 09:08:22.862224 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:08:22.862199    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7cc68bb_cf9c_468b_8658_186f8857d6a9.slice/crio-5e33a4e69eba3b112d19b7cfa637fb525fed9a5f47672a62fd82d46cdb88d4f5 WatchSource:0}: Error finding container 5e33a4e69eba3b112d19b7cfa637fb525fed9a5f47672a62fd82d46cdb88d4f5: Status 404 returned error can't find the container with id 5e33a4e69eba3b112d19b7cfa637fb525fed9a5f47672a62fd82d46cdb88d4f5
Sep 07 09:08:22.970831 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.970794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fc66649-hs5vz" event={"ID":"e7cc68bb-cf9c-468b-8658-186f8857d6a9","Type":"ContainerStarted","Data":"5e33a4e69eba3b112d19b7cfa637fb525fed9a5f47672a62fd82d46cdb88d4f5"}
Sep 07 09:08:22.972167 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:22.972137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-whncm" event={"ID":"32a66938-309e-44da-8c4a-07e1005cf4d6","Type":"ContainerStarted","Data":"ed1c44ef5675b278f49bbab9d212ec4436c4b69bf439b498cf76c6a1be6b8cb0"}
Sep 07 09:08:25.989568 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:25.989525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-whncm" event={"ID":"32a66938-309e-44da-8c4a-07e1005cf4d6","Type":"ContainerStarted","Data":"50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be"}
Sep 07 09:08:25.990062 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:25.989596    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:25.991139 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:25.991113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fc66649-hs5vz" event={"ID":"e7cc68bb-cf9c-468b-8658-186f8857d6a9","Type":"ContainerStarted","Data":"580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1"}
Sep 07 09:08:25.991248 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:25.991170    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:26.007119 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:26.007069    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6957b5f6b-whncm" podStartSLOduration=2.4898967020000002 podStartE2EDuration="5.007054086s" totalImagesPullingTime="2.517157384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:08:26.005801665 +0000 UTC m=+563.974972310" watchObservedRunningTime="2026-09-07 09:08:26.007054086 +0000 UTC m=+563.976224696"
Sep 07 09:08:26.022835 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:26.022789    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67fc66649-hs5vz" podStartSLOduration=2.944883833 podStartE2EDuration="5.022775288s" totalImagesPullingTime="2.077891455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:08:26.021126852 +0000 UTC m=+563.990297458" watchObservedRunningTime="2026-09-07 09:08:26.022775288 +0000 UTC m=+563.991945893"
Sep 07 09:08:36.999124 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:36.999091    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:08:36.999511 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:36.999169    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:37.063589 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.063554    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67fc66649-hs5vz"]
Sep 07 09:08:37.063817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.063785    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67fc66649-hs5vz" podUID="e7cc68bb-cf9c-468b-8658-186f8857d6a9" containerName="manager" containerID="cri-o://580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1" gracePeriod=30
Sep 07 09:08:37.307668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.307639    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:37.379695 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.379663    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-webhook-cert\") pod \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") "
Sep 07 09:08:37.379868 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.379789    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq6mj\" (UniqueName: \"kubernetes.io/projected/e7cc68bb-cf9c-468b-8658-186f8857d6a9-kube-api-access-hq6mj\") pod \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") "
Sep 07 09:08:37.379868 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.379819    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-metrics-certs\") pod \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\" (UID: \"e7cc68bb-cf9c-468b-8658-186f8857d6a9\") "
Sep 07 09:08:37.381951 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.381911    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e7cc68bb-cf9c-468b-8658-186f8857d6a9" (UID: "e7cc68bb-cf9c-468b-8658-186f8857d6a9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:08:37.381951 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.381919    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7cc68bb-cf9c-468b-8658-186f8857d6a9-kube-api-access-hq6mj" (OuterVolumeSpecName: "kube-api-access-hq6mj") pod "e7cc68bb-cf9c-468b-8658-186f8857d6a9" (UID: "e7cc68bb-cf9c-468b-8658-186f8857d6a9"). InnerVolumeSpecName "kube-api-access-hq6mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:08:37.382096 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.381926    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e7cc68bb-cf9c-468b-8658-186f8857d6a9" (UID: "e7cc68bb-cf9c-468b-8658-186f8857d6a9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:08:37.480370 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.480332    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq6mj\" (UniqueName: \"kubernetes.io/projected/e7cc68bb-cf9c-468b-8658-186f8857d6a9-kube-api-access-hq6mj\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:08:37.480370 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.480363    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:08:37.480370 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:37.480373    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7cc68bb-cf9c-468b-8658-186f8857d6a9-webhook-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:08:38.047366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.047325    2563 generic.go:356] "Generic (PLEG): container finished" podID="e7cc68bb-cf9c-468b-8658-186f8857d6a9" containerID="580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1" exitCode=0
Sep 07 09:08:38.047366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.047370    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fc66649-hs5vz"
Sep 07 09:08:38.047813 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.047419    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fc66649-hs5vz" event={"ID":"e7cc68bb-cf9c-468b-8658-186f8857d6a9","Type":"ContainerDied","Data":"580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1"}
Sep 07 09:08:38.047813 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.047467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fc66649-hs5vz" event={"ID":"e7cc68bb-cf9c-468b-8658-186f8857d6a9","Type":"ContainerDied","Data":"5e33a4e69eba3b112d19b7cfa637fb525fed9a5f47672a62fd82d46cdb88d4f5"}
Sep 07 09:08:38.047813 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.047491    2563 scope.go:117] "RemoveContainer" containerID="580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1"
Sep 07 09:08:38.056651 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.056632    2563 scope.go:117] "RemoveContainer" containerID="580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1"
Sep 07 09:08:38.056935 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:08:38.056915    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1\": container with ID starting with 580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1 not found: ID does not exist" containerID="580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1"
Sep 07 09:08:38.056988 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.056943    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1"} err="failed to get container status \"580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1\": rpc error: code = NotFound desc = could not find container \"580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1\": container with ID starting with 580d723341c9778623d6feb1edb574e65bb624fc785e5c1163ce0fc0fb7e79b1 not found: ID does not exist"
Sep 07 09:08:38.067968 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.067948    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67fc66649-hs5vz"]
Sep 07 09:08:38.071413 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.071394    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67fc66649-hs5vz"]
Sep 07 09:08:38.464889 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:38.464793    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7cc68bb-cf9c-468b-8658-186f8857d6a9" path="/var/lib/kubelet/pods/e7cc68bb-cf9c-468b-8658-186f8857d6a9/volumes"
Sep 07 09:08:59.346305 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.346248    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"]
Sep 07 09:08:59.346768 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.346615    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7cc68bb-cf9c-468b-8658-186f8857d6a9" containerName="manager"
Sep 07 09:08:59.346768 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.346626    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7cc68bb-cf9c-468b-8658-186f8857d6a9" containerName="manager"
Sep 07 09:08:59.346768 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.346688    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7cc68bb-cf9c-468b-8658-186f8857d6a9" containerName="manager"
Sep 07 09:08:59.350087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.350070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.354725 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.354700    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rckvc\""
Sep 07 09:08:59.354871 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.354818    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 09:08:59.354871 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.354823    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 07 09:08:59.360366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.360346    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"]
Sep 07 09:08:59.470801 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.470765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zttkw\" (UniqueName: \"kubernetes.io/projected/10ea0976-0725-4357-94ad-17791346359e-kube-api-access-zttkw\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.470953 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.470822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea0976-0725-4357-94ad-17791346359e-openshift-service-ca\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.470953 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.470871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-maas-api-tls\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.470953 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.470902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-metrics-certs\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.571666 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.571626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-maas-api-tls\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.571848 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.571685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-metrics-certs\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.571848 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.571716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zttkw\" (UniqueName: \"kubernetes.io/projected/10ea0976-0725-4357-94ad-17791346359e-kube-api-access-zttkw\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.571848 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.571769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea0976-0725-4357-94ad-17791346359e-openshift-service-ca\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.572432 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.572405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea0976-0725-4357-94ad-17791346359e-openshift-service-ca\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.574101 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.574076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-metrics-certs\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.574195 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.574112    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-maas-api-tls\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.580414 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.580388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zttkw\" (UniqueName: \"kubernetes.io/projected/10ea0976-0725-4357-94ad-17791346359e-kube-api-access-zttkw\") pod \"maas-api-5877fc59f9-rvlhb\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.661405 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.661314    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:08:59.707020 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.706982    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-s55z8"]
Sep 07 09:08:59.709711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.709674    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:08:59.715172 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.715079    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-w7cxx\""
Sep 07 09:08:59.715172 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.715098    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 07 09:08:59.726401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.726374    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-s55z8"]
Sep 07 09:08:59.874208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.874163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e49a562-3d44-40a0-a184-b90e5cd812dc-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-s55z8\" (UID: \"6e49a562-3d44-40a0-a184-b90e5cd812dc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:08:59.874208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.874208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhm4h\" (UniqueName: \"kubernetes.io/projected/6e49a562-3d44-40a0-a184-b90e5cd812dc-kube-api-access-rhm4h\") pod \"payload-processing-5b54b7fbb7-s55z8\" (UID: \"6e49a562-3d44-40a0-a184-b90e5cd812dc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:08:59.948569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.948534    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"]
Sep 07 09:08:59.950888 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:08:59.950855    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ea0976_0725_4357_94ad_17791346359e.slice/crio-f8c17875af342817e5500664cbb2fc4e1a8313898da5db3ff3466e60e18a1dfb WatchSource:0}: Error finding container f8c17875af342817e5500664cbb2fc4e1a8313898da5db3ff3466e60e18a1dfb: Status 404 returned error can't find the container with id f8c17875af342817e5500664cbb2fc4e1a8313898da5db3ff3466e60e18a1dfb
Sep 07 09:08:59.974836 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.974800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e49a562-3d44-40a0-a184-b90e5cd812dc-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-s55z8\" (UID: \"6e49a562-3d44-40a0-a184-b90e5cd812dc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:08:59.974836 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.974836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhm4h\" (UniqueName: \"kubernetes.io/projected/6e49a562-3d44-40a0-a184-b90e5cd812dc-kube-api-access-rhm4h\") pod \"payload-processing-5b54b7fbb7-s55z8\" (UID: \"6e49a562-3d44-40a0-a184-b90e5cd812dc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:08:59.975574 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.975547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e49a562-3d44-40a0-a184-b90e5cd812dc-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-s55z8\" (UID: \"6e49a562-3d44-40a0-a184-b90e5cd812dc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:08:59.982513 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:08:59.982484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhm4h\" (UniqueName: \"kubernetes.io/projected/6e49a562-3d44-40a0-a184-b90e5cd812dc-kube-api-access-rhm4h\") pod \"payload-processing-5b54b7fbb7-s55z8\" (UID: \"6e49a562-3d44-40a0-a184-b90e5cd812dc\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:09:00.032082 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:00.032057    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:09:00.164763 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:00.164730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb" event={"ID":"10ea0976-0725-4357-94ad-17791346359e","Type":"ContainerStarted","Data":"f8c17875af342817e5500664cbb2fc4e1a8313898da5db3ff3466e60e18a1dfb"}
Sep 07 09:09:00.301229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:00.301194    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-s55z8"]
Sep 07 09:09:00.304109 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:09:00.304083    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e49a562_3d44_40a0_a184_b90e5cd812dc.slice/crio-5c9cae3f5dc2a84c0a3990ef193274c404c3da61ed1e9641beaf6901db1ce188 WatchSource:0}: Error finding container 5c9cae3f5dc2a84c0a3990ef193274c404c3da61ed1e9641beaf6901db1ce188: Status 404 returned error can't find the container with id 5c9cae3f5dc2a84c0a3990ef193274c404c3da61ed1e9641beaf6901db1ce188
Sep 07 09:09:01.173756 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:01.173718    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8" event={"ID":"6e49a562-3d44-40a0-a184-b90e5cd812dc","Type":"ContainerStarted","Data":"5c9cae3f5dc2a84c0a3990ef193274c404c3da61ed1e9641beaf6901db1ce188"}
Sep 07 09:09:04.191039 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:04.190992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb" event={"ID":"10ea0976-0725-4357-94ad-17791346359e","Type":"ContainerStarted","Data":"a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033"}
Sep 07 09:09:04.191543 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:04.191062    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:09:04.192635 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:04.192610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8" event={"ID":"6e49a562-3d44-40a0-a184-b90e5cd812dc","Type":"ContainerStarted","Data":"05c02a35a37f76f92fa7b0d8bbefdab1d1b9b2795ddab14e9b1fe2f13adbba78"}
Sep 07 09:09:04.192753 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:04.192738    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:09:04.207732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:04.207683    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb" podStartSLOduration=2.00318271 podStartE2EDuration="5.207668117s" totalImagesPullingTime="3.204485407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:09:04.206648522 +0000 UTC m=+602.175819140" watchObservedRunningTime="2026-09-07 09:09:04.207668117 +0000 UTC m=+602.176838722"
Sep 07 09:09:04.222829 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:04.222772    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8" podStartSLOduration=2.365618989 podStartE2EDuration="5.222757815s" totalImagesPullingTime="2.857138826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:09:04.221399635 +0000 UTC m=+602.190570240" watchObservedRunningTime="2026-09-07 09:09:04.222757815 +0000 UTC m=+602.191928420"
Sep 07 09:09:10.202687 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:10.202658    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:09:15.200347 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:09:15.200277    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-s55z8"
Sep 07 09:12:28.172438 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.172357    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5fcc76f457-qkpsr"]
Sep 07 09:12:28.174729 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.174709    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fcc76f457-qkpsr"
Sep 07 09:12:28.182926 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.182897    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fcc76f457-qkpsr"]
Sep 07 09:12:28.298123 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.298084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b66812ef-287c-4024-b311-e00e51e7f23c-tls-cert\") pod \"authorino-5fcc76f457-qkpsr\" (UID: \"b66812ef-287c-4024-b311-e00e51e7f23c\") " pod="kuadrant-system/authorino-5fcc76f457-qkpsr"
Sep 07 09:12:28.298123 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.298127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8zrl\" (UniqueName: \"kubernetes.io/projected/b66812ef-287c-4024-b311-e00e51e7f23c-kube-api-access-h8zrl\") pod \"authorino-5fcc76f457-qkpsr\" (UID: \"b66812ef-287c-4024-b311-e00e51e7f23c\") " pod="kuadrant-system/authorino-5fcc76f457-qkpsr"
Sep 07 09:12:28.398909 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.398870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b66812ef-287c-4024-b311-e00e51e7f23c-tls-cert\") pod \"authorino-5fcc76f457-qkpsr\" (UID: \"b66812ef-287c-4024-b311-e00e51e7f23c\") " pod="kuadrant-system/authorino-5fcc76f457-qkpsr"
Sep 07 09:12:28.398909 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.398909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8zrl\" (UniqueName: \"kubernetes.io/projected/b66812ef-287c-4024-b311-e00e51e7f23c-kube-api-access-h8zrl\") pod \"authorino-5fcc76f457-qkpsr\" (UID: \"b66812ef-287c-4024-b311-e00e51e7f23c\") " pod="kuadrant-system/authorino-5fcc76f457-qkpsr"
Sep 07 09:12:28.401302 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.401263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b66812ef-287c-4024-b311-e00e51e7f23c-tls-cert\") pod \"authorino-5fcc76f457-qkpsr\" (UID: \"b66812ef-287c-4024-b311-e00e51e7f23c\") " pod="kuadrant-system/authorino-5fcc76f457-qkpsr"
Sep 07 09:12:28.408220 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.408192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8zrl\" (UniqueName: \"kubernetes.io/projected/b66812ef-287c-4024-b311-e00e51e7f23c-kube-api-access-h8zrl\") pod \"authorino-5fcc76f457-qkpsr\" (UID: \"b66812ef-287c-4024-b311-e00e51e7f23c\") " pod="kuadrant-system/authorino-5fcc76f457-qkpsr"
Sep 07 09:12:28.484541 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.484461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fcc76f457-qkpsr"
Sep 07 09:12:28.757193 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.757168    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fcc76f457-qkpsr"]
Sep 07 09:12:28.759256 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:12:28.759231    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb66812ef_287c_4024_b311_e00e51e7f23c.slice/crio-44ed53f0fcc4d9d3698e5be561b12a183671e79a7e79eba77c0f8fd3f0de2de4 WatchSource:0}: Error finding container 44ed53f0fcc4d9d3698e5be561b12a183671e79a7e79eba77c0f8fd3f0de2de4: Status 404 returned error can't find the container with id 44ed53f0fcc4d9d3698e5be561b12a183671e79a7e79eba77c0f8fd3f0de2de4
Sep 07 09:12:28.760662 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:28.760645    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:12:29.130497 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:29.130405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fcc76f457-qkpsr" event={"ID":"b66812ef-287c-4024-b311-e00e51e7f23c","Type":"ContainerStarted","Data":"44ed53f0fcc4d9d3698e5be561b12a183671e79a7e79eba77c0f8fd3f0de2de4"}
Sep 07 09:12:30.145721 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.145674    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fcc76f457-qkpsr" event={"ID":"b66812ef-287c-4024-b311-e00e51e7f23c","Type":"ContainerStarted","Data":"f98c38670adeaa3d28c087198c2a34973fa1e32506fc1014d0b47da87b887889"}
Sep 07 09:12:30.164672 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.164611    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5fcc76f457-qkpsr" podStartSLOduration=1.508512528 podStartE2EDuration="2.164594198s" totalImagesPullingTime="656.08167ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:12:30.162217446 +0000 UTC m=+808.131388052" watchObservedRunningTime="2026-09-07 09:12:30.164594198 +0000 UTC m=+808.133764802"
Sep 07 09:12:30.192120 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.192078    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7989bc5db8-pj9g2"]
Sep 07 09:12:30.192372 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.192339    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7989bc5db8-pj9g2" podUID="5ac5f019-c809-4793-9214-1033bc8c0a7b" containerName="authorino" containerID="cri-o://41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3" gracePeriod=30
Sep 07 09:12:30.450835 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.450809    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:12:30.621892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.621854    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac5f019-c809-4793-9214-1033bc8c0a7b-tls-cert\") pod \"5ac5f019-c809-4793-9214-1033bc8c0a7b\" (UID: \"5ac5f019-c809-4793-9214-1033bc8c0a7b\") "
Sep 07 09:12:30.622077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.621906    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtg6d\" (UniqueName: \"kubernetes.io/projected/5ac5f019-c809-4793-9214-1033bc8c0a7b-kube-api-access-xtg6d\") pod \"5ac5f019-c809-4793-9214-1033bc8c0a7b\" (UID: \"5ac5f019-c809-4793-9214-1033bc8c0a7b\") "
Sep 07 09:12:30.623967 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.623937    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac5f019-c809-4793-9214-1033bc8c0a7b-kube-api-access-xtg6d" (OuterVolumeSpecName: "kube-api-access-xtg6d") pod "5ac5f019-c809-4793-9214-1033bc8c0a7b" (UID: "5ac5f019-c809-4793-9214-1033bc8c0a7b"). InnerVolumeSpecName "kube-api-access-xtg6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:12:30.633662 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.633630    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac5f019-c809-4793-9214-1033bc8c0a7b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5ac5f019-c809-4793-9214-1033bc8c0a7b" (UID: "5ac5f019-c809-4793-9214-1033bc8c0a7b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:12:30.723241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.723163    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ac5f019-c809-4793-9214-1033bc8c0a7b-tls-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:12:30.723241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:30.723194    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtg6d\" (UniqueName: \"kubernetes.io/projected/5ac5f019-c809-4793-9214-1033bc8c0a7b-kube-api-access-xtg6d\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:12:31.151754 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:31.151719    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ac5f019-c809-4793-9214-1033bc8c0a7b" containerID="41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3" exitCode=0
Sep 07 09:12:31.152238 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:31.151773    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7989bc5db8-pj9g2"
Sep 07 09:12:31.152238 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:31.151803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7989bc5db8-pj9g2" event={"ID":"5ac5f019-c809-4793-9214-1033bc8c0a7b","Type":"ContainerDied","Data":"41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3"}
Sep 07 09:12:31.152238 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:31.151846    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7989bc5db8-pj9g2" event={"ID":"5ac5f019-c809-4793-9214-1033bc8c0a7b","Type":"ContainerDied","Data":"f0237b5f8ac676c883d256a6c87a456b1cc254f223af7112851df3a8528ea83c"}
Sep 07 09:12:31.152238 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:31.151864    2563 scope.go:117] "RemoveContainer" containerID="41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3"
Sep 07 09:12:31.161992 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:31.161966    2563 scope.go:117] "RemoveContainer" containerID="41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3"
Sep 07 09:12:31.162334 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:12:31.162313    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3\": container with ID starting with 41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3 not found: ID does not exist" containerID="41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3"
Sep 07 09:12:31.162404 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:31.162344    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3"} err="failed to get container status \"41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3\": rpc error: code = NotFound desc = could not find container \"41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3\": container with ID starting with 41801237d5c3666a37f735d4925fae53b10b2b6d7734d8ec3fd512d64aaf1ba3 not found: ID does not exist"
Sep 07 09:12:31.173018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:31.172992    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7989bc5db8-pj9g2"]
Sep 07 09:12:31.176955 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:31.176929    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7989bc5db8-pj9g2"]
Sep 07 09:12:32.464196 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:32.464163    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ac5f019-c809-4793-9214-1033bc8c0a7b" path="/var/lib/kubelet/pods/5ac5f019-c809-4793-9214-1033bc8c0a7b/volumes"
Sep 07 09:12:47.613371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.613333    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"]
Sep 07 09:12:47.613739 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.613683    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac5f019-c809-4793-9214-1033bc8c0a7b" containerName="authorino"
Sep 07 09:12:47.613739 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.613696    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac5f019-c809-4793-9214-1033bc8c0a7b" containerName="authorino"
Sep 07 09:12:47.613842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.613752    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac5f019-c809-4793-9214-1033bc8c0a7b" containerName="authorino"
Sep 07 09:12:47.616609 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.616588    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.619352 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.619327    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 09:12:47.619491 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.619328    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-614a02-openshift-default-dockercfg-mqvdt\""
Sep 07 09:12:47.629759 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.629728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"]
Sep 07 09:12:47.669861 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.669817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-certs\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.669861 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.669861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-data\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.670118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.669881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a709fe-0870-4529-905c-5075b7758948-istiod-ca-cert\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.670118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.669943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ztrv\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-kube-api-access-6ztrv\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.670118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.669967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-envoy\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.670118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.670014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-credential-socket\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.670118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.670037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-istio-token\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.670118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.670067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a709fe-0870-4529-905c-5075b7758948-istio-podinfo\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.670421 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.670137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-socket\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.771453 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-certs\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.771453 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-data\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.771715 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a709fe-0870-4529-905c-5075b7758948-istiod-ca-cert\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.771715 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ztrv\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-kube-api-access-6ztrv\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.771715 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-envoy\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.771715 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-credential-socket\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.771715 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-istio-token\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.771715 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a709fe-0870-4529-905c-5075b7758948-istio-podinfo\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.772038 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-socket\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.772038 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.771910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-certs\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.772038 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.772024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-credential-socket\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.772186 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.772086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-socket\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.772492 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.772219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-data\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.772744 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.772725    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a709fe-0870-4529-905c-5075b7758948-istiod-ca-cert\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.774208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.774185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-envoy\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.774350 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.774334    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a709fe-0870-4529-905c-5075b7758948-istio-podinfo\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.780320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.780268    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ztrv\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-kube-api-access-6ztrv\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.782024 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.781996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-istio-token\") pod \"e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") " pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:47.929215 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:47.929130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:48.208761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:48.208676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"]
Sep 07 09:12:48.211986 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:12:48.211959    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a709fe_0870_4529_905c_5075b7758948.slice/crio-a1798012d6be10482a180e712f10e5e89787c99d37dd914b65dba09253ec3804 WatchSource:0}: Error finding container a1798012d6be10482a180e712f10e5e89787c99d37dd914b65dba09253ec3804: Status 404 returned error can't find the container with id a1798012d6be10482a180e712f10e5e89787c99d37dd914b65dba09253ec3804
Sep 07 09:12:48.230380 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:48.230349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" event={"ID":"26a709fe-0870-4529-905c-5075b7758948","Type":"ContainerStarted","Data":"a1798012d6be10482a180e712f10e5e89787c99d37dd914b65dba09253ec3804"}
Sep 07 09:12:49.193674 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:49.193635    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:49.196627 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:49.196599    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:49.199322 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:49.199295    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ckkfv\""
Sep 07 09:12:49.203479 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:49.203441    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:49.288394 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:49.288350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j9sf\" (UniqueName: \"kubernetes.io/projected/8a204ea7-02d8-4d89-91c9-166992a9216f-kube-api-access-7j9sf\") pod \"test-curl-7993-140128886255280\" (UID: \"8a204ea7-02d8-4d89-91c9-166992a9216f\") " pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:49.388963 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:49.388919    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j9sf\" (UniqueName: \"kubernetes.io/projected/8a204ea7-02d8-4d89-91c9-166992a9216f-kube-api-access-7j9sf\") pod \"test-curl-7993-140128886255280\" (UID: \"8a204ea7-02d8-4d89-91c9-166992a9216f\") " pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:49.398096 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:49.398049    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j9sf\" (UniqueName: \"kubernetes.io/projected/8a204ea7-02d8-4d89-91c9-166992a9216f-kube-api-access-7j9sf\") pod \"test-curl-7993-140128886255280\" (UID: \"8a204ea7-02d8-4d89-91c9-166992a9216f\") " pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:49.512679 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:49.512643    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:49.791965 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:49.791909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:49.794752 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:12:49.794719    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a204ea7_02d8_4d89_91c9_166992a9216f.slice/crio-ebda5aadb7ee8df05c66de56222514d724a6ecc6abd3a70c16893cb180a56e4b WatchSource:0}: Error finding container ebda5aadb7ee8df05c66de56222514d724a6ecc6abd3a70c16893cb180a56e4b: Status 404 returned error can't find the container with id ebda5aadb7ee8df05c66de56222514d724a6ecc6abd3a70c16893cb180a56e4b
Sep 07 09:12:50.243664 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:50.243550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7993-140128886255280" event={"ID":"8a204ea7-02d8-4d89-91c9-166992a9216f","Type":"ContainerStarted","Data":"ebda5aadb7ee8df05c66de56222514d724a6ecc6abd3a70c16893cb180a56e4b"}
Sep 07 09:12:51.161363 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:51.161328    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:12:51.161432 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:51.161400    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:12:51.161467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:51.161430    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:12:51.251237 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:51.251197    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7993-140128886255280" event={"ID":"8a204ea7-02d8-4d89-91c9-166992a9216f","Type":"ContainerStarted","Data":"8741093d0870aacbb6359381638f69b4b2eec600eb0f0ccfc524508ea27df8bd"}
Sep 07 09:12:51.253043 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:51.253010    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" event={"ID":"26a709fe-0870-4529-905c-5075b7758948","Type":"ContainerStarted","Data":"29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084"}
Sep 07 09:12:51.266064 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:51.264650    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7993-140128886255280" podStartSLOduration=0.903865603 podStartE2EDuration="2.264604333s" totalImagesPullingTime="1.36073873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:12:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:12:51.264351489 +0000 UTC m=+829.233522095" watchObservedRunningTime="2026-09-07 09:12:51.264604333 +0000 UTC m=+829.233774937"
Sep 07 09:12:51.285731 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:51.285680    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" podStartSLOduration=1.339169062 podStartE2EDuration="4.285662124s" totalImagesPullingTime="2.946493062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:12:51.283671547 +0000 UTC m=+829.252842175" watchObservedRunningTime="2026-09-07 09:12:51.285662124 +0000 UTC m=+829.254832729"
Sep 07 09:12:51.367842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:51.367761    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7993-140128886255280_8a204ea7-02d8-4d89-91c9-166992a9216f/test-curl-7993-140128886255280/0.log"
Sep 07 09:12:51.930026 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:51.929997    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:52.259099 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:52.259062    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a204ea7-02d8-4d89-91c9-166992a9216f" containerID="8741093d0870aacbb6359381638f69b4b2eec600eb0f0ccfc524508ea27df8bd" exitCode=0
Sep 07 09:12:52.259454 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:52.259146    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7993-140128886255280" event={"ID":"8a204ea7-02d8-4d89-91c9-166992a9216f","Type":"ContainerDied","Data":"8741093d0870aacbb6359381638f69b4b2eec600eb0f0ccfc524508ea27df8bd"}
Sep 07 09:12:52.934668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:52.934635    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:53.264869 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.264839    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:53.265739 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.265720    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:12:53.403967 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.403942    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:53.436583 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.436551    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:53.442328 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.442299    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:53.533399 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.533273    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j9sf\" (UniqueName: \"kubernetes.io/projected/8a204ea7-02d8-4d89-91c9-166992a9216f-kube-api-access-7j9sf\") pod \"8a204ea7-02d8-4d89-91c9-166992a9216f\" (UID: \"8a204ea7-02d8-4d89-91c9-166992a9216f\") "
Sep 07 09:12:53.535404 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.535379    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a204ea7-02d8-4d89-91c9-166992a9216f-kube-api-access-7j9sf" (OuterVolumeSpecName: "kube-api-access-7j9sf") pod "8a204ea7-02d8-4d89-91c9-166992a9216f" (UID: "8a204ea7-02d8-4d89-91c9-166992a9216f"). InnerVolumeSpecName "kube-api-access-7j9sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:12:53.555319 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.555272    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:53.555683 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.555668    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a204ea7-02d8-4d89-91c9-166992a9216f" containerName="test-curl-7993-140128886255280"
Sep 07 09:12:53.555731 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.555685    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a204ea7-02d8-4d89-91c9-166992a9216f" containerName="test-curl-7993-140128886255280"
Sep 07 09:12:53.555774 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.555740    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a204ea7-02d8-4d89-91c9-166992a9216f" containerName="test-curl-7993-140128886255280"
Sep 07 09:12:53.557747 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.557724    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:53.568128 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.568097    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:53.634006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.633966    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swtgj\" (UniqueName: \"kubernetes.io/projected/5f81aafd-d891-43ff-83db-c18e76e930a8-kube-api-access-swtgj\") pod \"test-curl-7993-140128886255280\" (UID: \"5f81aafd-d891-43ff-83db-c18e76e930a8\") " pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:53.634172 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.634025    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j9sf\" (UniqueName: \"kubernetes.io/projected/8a204ea7-02d8-4d89-91c9-166992a9216f-kube-api-access-7j9sf\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:12:53.734981 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.734937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtgj\" (UniqueName: \"kubernetes.io/projected/5f81aafd-d891-43ff-83db-c18e76e930a8-kube-api-access-swtgj\") pod \"test-curl-7993-140128886255280\" (UID: \"5f81aafd-d891-43ff-83db-c18e76e930a8\") " pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:53.742968 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.742939    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swtgj\" (UniqueName: \"kubernetes.io/projected/5f81aafd-d891-43ff-83db-c18e76e930a8-kube-api-access-swtgj\") pod \"test-curl-7993-140128886255280\" (UID: \"5f81aafd-d891-43ff-83db-c18e76e930a8\") " pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:53.871427 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:53.871339    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:54.141692 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.141651    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:54.144009 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:12:54.143979    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f81aafd_d891_43ff_83db_c18e76e930a8.slice/crio-426cc0255183930e570b5f0bdff9c1f4d39dabaa2649655dc5f7f3ef3d78afef WatchSource:0}: Error finding container 426cc0255183930e570b5f0bdff9c1f4d39dabaa2649655dc5f7f3ef3d78afef: Status 404 returned error can't find the container with id 426cc0255183930e570b5f0bdff9c1f4d39dabaa2649655dc5f7f3ef3d78afef
Sep 07 09:12:54.269799 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.269763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7993-140128886255280" event={"ID":"5f81aafd-d891-43ff-83db-c18e76e930a8","Type":"ContainerStarted","Data":"426cc0255183930e570b5f0bdff9c1f4d39dabaa2649655dc5f7f3ef3d78afef"}
Sep 07 09:12:54.270996 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.270969    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:54.270996 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.270993    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebda5aadb7ee8df05c66de56222514d724a6ecc6abd3a70c16893cb180a56e4b"
Sep 07 09:12:54.282485 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.282448    2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7993-140128886255280" oldPodUID="8a204ea7-02d8-4d89-91c9-166992a9216f" podUID="5f81aafd-d891-43ff-83db-c18e76e930a8"
Sep 07 09:12:54.453733 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.453655    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"]
Sep 07 09:12:54.457065 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.457043    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.459940 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.459913    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-614a02-serving-cert\""
Sep 07 09:12:54.465945 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.465220    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a204ea7-02d8-4d89-91c9-166992a9216f" path="/var/lib/kubelet/pods/8a204ea7-02d8-4d89-91c9-166992a9216f/volumes"
Sep 07 09:12:54.468742 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.468717    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"]
Sep 07 09:12:54.541611 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.541582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-metrics-certs\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.541732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.541648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.541732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.541715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gthdl\" (UniqueName: \"kubernetes.io/projected/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-kube-api-access-gthdl\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.541838 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.541806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-maas-api-tls\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.642768 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.642717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-metrics-certs\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.642950 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.642787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.642950 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.642826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gthdl\" (UniqueName: \"kubernetes.io/projected/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-kube-api-access-gthdl\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.642950 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.642878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-maas-api-tls\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.643723 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.643696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.645831 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.645809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-maas-api-tls\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.645928 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.645853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-metrics-certs\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.651519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.651499    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gthdl\" (UniqueName: \"kubernetes.io/projected/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-kube-api-access-gthdl\") pod \"maas-api-e2e-inf-a-614a02-7b449b79f-n7j92\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:54.770596 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:54.770567    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:55.059109 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.059046    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"]
Sep 07 09:12:55.061644 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:12:55.061611    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cbe9963_8eb1_4f8b_8de3_b8b10013f42e.slice/crio-550b23fe5705c9c23b821de2a66785ac633c2ea148d6d34d40f75b67d5256f7d WatchSource:0}: Error finding container 550b23fe5705c9c23b821de2a66785ac633c2ea148d6d34d40f75b67d5256f7d: Status 404 returned error can't find the container with id 550b23fe5705c9c23b821de2a66785ac633c2ea148d6d34d40f75b67d5256f7d
Sep 07 09:12:55.113006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.112968    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"]
Sep 07 09:12:55.115969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.115933    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:12:55.118842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.118822    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-614a02-dockercfg-l2hv5\""
Sep 07 09:12:55.119348 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.119328    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-614a02\""
Sep 07 09:12:55.132871 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.132845    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"]
Sep 07 09:12:55.248079 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.248034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fa1d525-101d-41af-9391-8438ee1d0fe0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f\" (UID: \"7fa1d525-101d-41af-9391-8438ee1d0fe0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:12:55.248258 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.248111    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw2ts\" (UniqueName: \"kubernetes.io/projected/7fa1d525-101d-41af-9391-8438ee1d0fe0-kube-api-access-xw2ts\") pod \"payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f\" (UID: \"7fa1d525-101d-41af-9391-8438ee1d0fe0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:12:55.278823 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.278784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92" event={"ID":"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e","Type":"ContainerStarted","Data":"550b23fe5705c9c23b821de2a66785ac633c2ea148d6d34d40f75b67d5256f7d"}
Sep 07 09:12:55.280549 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.280518    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f81aafd-d891-43ff-83db-c18e76e930a8" containerID="a3c419bfa721431b6f792d94c7633f72214842039a47b3c4d7066871a14f6859" exitCode=0
Sep 07 09:12:55.280684 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.280597    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7993-140128886255280" event={"ID":"5f81aafd-d891-43ff-83db-c18e76e930a8","Type":"ContainerDied","Data":"a3c419bfa721431b6f792d94c7633f72214842039a47b3c4d7066871a14f6859"}
Sep 07 09:12:55.348946 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.348846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fa1d525-101d-41af-9391-8438ee1d0fe0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f\" (UID: \"7fa1d525-101d-41af-9391-8438ee1d0fe0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:12:55.349124 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.348976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw2ts\" (UniqueName: \"kubernetes.io/projected/7fa1d525-101d-41af-9391-8438ee1d0fe0-kube-api-access-xw2ts\") pod \"payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f\" (UID: \"7fa1d525-101d-41af-9391-8438ee1d0fe0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:12:55.349625 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.349602    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fa1d525-101d-41af-9391-8438ee1d0fe0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f\" (UID: \"7fa1d525-101d-41af-9391-8438ee1d0fe0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:12:55.357495 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.357466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw2ts\" (UniqueName: \"kubernetes.io/projected/7fa1d525-101d-41af-9391-8438ee1d0fe0-kube-api-access-xw2ts\") pod \"payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f\" (UID: \"7fa1d525-101d-41af-9391-8438ee1d0fe0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:12:55.426607 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.426562    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:12:55.701342 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:55.701307    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"]
Sep 07 09:12:55.702567 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:12:55.702534    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa1d525_101d_41af_9391_8438ee1d0fe0.slice/crio-d638c41fd0189c54f74e120f1432f40f14e11fd44ae925a48a1ccfc1a09a0cc4 WatchSource:0}: Error finding container d638c41fd0189c54f74e120f1432f40f14e11fd44ae925a48a1ccfc1a09a0cc4: Status 404 returned error can't find the container with id d638c41fd0189c54f74e120f1432f40f14e11fd44ae925a48a1ccfc1a09a0cc4
Sep 07 09:12:56.288070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.288030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92" event={"ID":"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e","Type":"ContainerStarted","Data":"a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290"}
Sep 07 09:12:56.288564 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.288114    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:12:56.289868 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.289840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f" event={"ID":"7fa1d525-101d-41af-9391-8438ee1d0fe0","Type":"ContainerStarted","Data":"242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a"}
Sep 07 09:12:56.289984 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.289890    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f" event={"ID":"7fa1d525-101d-41af-9391-8438ee1d0fe0","Type":"ContainerStarted","Data":"d638c41fd0189c54f74e120f1432f40f14e11fd44ae925a48a1ccfc1a09a0cc4"}
Sep 07 09:12:56.307933 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.307866    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92" podStartSLOduration=1.860415881 podStartE2EDuration="2.30784812s" totalImagesPullingTime="447.432239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:12:56.306130731 +0000 UTC m=+834.275301337" watchObservedRunningTime="2026-09-07 09:12:56.30784812 +0000 UTC m=+834.277018740"
Sep 07 09:12:56.444080 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.444053    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:56.463821 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.463646    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f" podStartSLOduration=1.463623976 podStartE2EDuration="1.463623976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:12:56.326069622 +0000 UTC m=+834.295240228" watchObservedRunningTime="2026-09-07 09:12:56.463623976 +0000 UTC m=+834.432794582"
Sep 07 09:12:56.467207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.467121    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7993-140128886255280_5f81aafd-d891-43ff-83db-c18e76e930a8/test-curl-7993-140128886255280/0.log"
Sep 07 09:12:56.498664 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.498601    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:56.500512 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.500485    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7993-140128886255280"]
Sep 07 09:12:56.560878 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.560775    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swtgj\" (UniqueName: \"kubernetes.io/projected/5f81aafd-d891-43ff-83db-c18e76e930a8-kube-api-access-swtgj\") pod \"5f81aafd-d891-43ff-83db-c18e76e930a8\" (UID: \"5f81aafd-d891-43ff-83db-c18e76e930a8\") "
Sep 07 09:12:56.562855 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.562827    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f81aafd-d891-43ff-83db-c18e76e930a8-kube-api-access-swtgj" (OuterVolumeSpecName: "kube-api-access-swtgj") pod "5f81aafd-d891-43ff-83db-c18e76e930a8" (UID: "5f81aafd-d891-43ff-83db-c18e76e930a8"). InnerVolumeSpecName "kube-api-access-swtgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:12:56.661791 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:56.661756    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swtgj\" (UniqueName: \"kubernetes.io/projected/5f81aafd-d891-43ff-83db-c18e76e930a8-kube-api-access-swtgj\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:12:57.296156 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:57.296118    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:57.296156 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:57.296144    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="426cc0255183930e570b5f0bdff9c1f4d39dabaa2649655dc5f7f3ef3d78afef"
Sep 07 09:12:57.296717 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:57.296532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:12:57.306272 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:12:57.306240    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7993-140128886255280\" is forbidden: User \"system:node:ip-10-0-138-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-157.ec2.internal' and this object" podUID="5f81aafd-d891-43ff-83db-c18e76e930a8" pod="opendatahub/test-curl-7993-140128886255280"
Sep 07 09:12:58.465083 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:58.465049    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f81aafd-d891-43ff-83db-c18e76e930a8" path="/var/lib/kubelet/pods/5f81aafd-d891-43ff-83db-c18e76e930a8/volumes"
Sep 07 09:12:59.618166 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.618126    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"]
Sep 07 09:12:59.618790 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.618767    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f81aafd-d891-43ff-83db-c18e76e930a8" containerName="test-curl-7993-140128886255280"
Sep 07 09:12:59.618853 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.618793    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f81aafd-d891-43ff-83db-c18e76e930a8" containerName="test-curl-7993-140128886255280"
Sep 07 09:12:59.618924 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.618911    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f81aafd-d891-43ff-83db-c18e76e930a8" containerName="test-curl-7993-140128886255280"
Sep 07 09:12:59.621525 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.621502    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.624055 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.624029    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-140be7ab-serving-cert\""
Sep 07 09:12:59.632677 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.632649    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"]
Sep 07 09:12:59.691323 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.691278    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmmn7\" (UniqueName: \"kubernetes.io/projected/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-kube-api-access-hmmn7\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.691570 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.691361    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-openshift-service-ca\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.691570 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.691387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-maas-api-tls\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.691570 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.691426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-metrics-certs\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.792615 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.792581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-openshift-service-ca\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.792828 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.792622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-maas-api-tls\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.792828 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.792652    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-metrics-certs\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.792828 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:12:59.792757    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-serving-cert: secret "maas-api-e2e-ait-140be7ab-serving-cert" not found
Sep 07 09:12:59.792828 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:12:59.792827    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-maas-api-tls podName:4db4cfcd-eaab-4bf4-be1b-c9c79daad990 nodeName:}" failed. No retries permitted until 2026-09-07 09:13:00.292805521 +0000 UTC m=+838.261976125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-maas-api-tls") pod "maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz" (UID: "4db4cfcd-eaab-4bf4-be1b-c9c79daad990") : secret "maas-api-e2e-ait-140be7ab-serving-cert" not found
Sep 07 09:12:59.793078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.792847    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmmn7\" (UniqueName: \"kubernetes.io/projected/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-kube-api-access-hmmn7\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.793381 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.793356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-openshift-service-ca\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.795177 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.795158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-metrics-certs\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:12:59.801317 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:12:59.801268    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmmn7\" (UniqueName: \"kubernetes.io/projected/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-kube-api-access-hmmn7\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:13:00.288390 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.288355    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"]
Sep 07 09:13:00.291015 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.290996    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:00.293830 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.293805    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-140be7ab\""
Sep 07 09:13:00.293970 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.293865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-140be7ab-dockercfg-wphzs\""
Sep 07 09:13:00.298310 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.298106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-maas-api-tls\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:13:00.301214 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.301188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-maas-api-tls\") pod \"maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:13:00.301737 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.301719    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"]
Sep 07 09:13:00.319079 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.319048    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"]
Sep 07 09:13:00.322523 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.322501    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:00.330490 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.330462    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"]
Sep 07 09:13:00.399119 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.399088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znf7h\" (UniqueName: \"kubernetes.io/projected/b32a3f9a-748d-4663-865a-d9ea2237192a-kube-api-access-znf7h\") pod \"payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c\" (UID: \"b32a3f9a-748d-4663-865a-d9ea2237192a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:00.399119 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.399124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b32a3f9a-748d-4663-865a-d9ea2237192a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c\" (UID: \"b32a3f9a-748d-4663-865a-d9ea2237192a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:00.399379 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.399142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kztr2\" (UniqueName: \"kubernetes.io/projected/4f28cb92-730b-4e70-967e-d94c96e6ca87-kube-api-access-kztr2\") pod \"payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn\" (UID: \"4f28cb92-730b-4e70-967e-d94c96e6ca87\") " pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:00.399379 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.399341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f28cb92-730b-4e70-967e-d94c96e6ca87-plugins-config-volume\") pod \"payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn\" (UID: \"4f28cb92-730b-4e70-967e-d94c96e6ca87\") " pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:00.500879 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.500843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f28cb92-730b-4e70-967e-d94c96e6ca87-plugins-config-volume\") pod \"payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn\" (UID: \"4f28cb92-730b-4e70-967e-d94c96e6ca87\") " pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:00.501064 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.500910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znf7h\" (UniqueName: \"kubernetes.io/projected/b32a3f9a-748d-4663-865a-d9ea2237192a-kube-api-access-znf7h\") pod \"payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c\" (UID: \"b32a3f9a-748d-4663-865a-d9ea2237192a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:00.501064 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.500934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b32a3f9a-748d-4663-865a-d9ea2237192a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c\" (UID: \"b32a3f9a-748d-4663-865a-d9ea2237192a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:00.501064 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.500952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kztr2\" (UniqueName: \"kubernetes.io/projected/4f28cb92-730b-4e70-967e-d94c96e6ca87-kube-api-access-kztr2\") pod \"payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn\" (UID: \"4f28cb92-730b-4e70-967e-d94c96e6ca87\") " pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:00.501849 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.501822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b32a3f9a-748d-4663-865a-d9ea2237192a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c\" (UID: \"b32a3f9a-748d-4663-865a-d9ea2237192a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:00.501927 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.501822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f28cb92-730b-4e70-967e-d94c96e6ca87-plugins-config-volume\") pod \"payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn\" (UID: \"4f28cb92-730b-4e70-967e-d94c96e6ca87\") " pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:00.508718 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.508689    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znf7h\" (UniqueName: \"kubernetes.io/projected/b32a3f9a-748d-4663-865a-d9ea2237192a-kube-api-access-znf7h\") pod \"payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c\" (UID: \"b32a3f9a-748d-4663-865a-d9ea2237192a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:00.508844 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.508821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kztr2\" (UniqueName: \"kubernetes.io/projected/4f28cb92-730b-4e70-967e-d94c96e6ca87-kube-api-access-kztr2\") pod \"payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn\" (UID: \"4f28cb92-730b-4e70-967e-d94c96e6ca87\") " pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:00.534129 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.534102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:13:00.602765 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.602719    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:00.635440 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.635389    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:00.811147 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.810137    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"]
Sep 07 09:13:00.816526 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:13:00.816494    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db4cfcd_eaab_4bf4_be1b_c9c79daad990.slice/crio-a37e651907fd919cc3b7f8639c9256ec28fba23c1e0b2d2dbdec5657d99495a1 WatchSource:0}: Error finding container a37e651907fd919cc3b7f8639c9256ec28fba23c1e0b2d2dbdec5657d99495a1: Status 404 returned error can't find the container with id a37e651907fd919cc3b7f8639c9256ec28fba23c1e0b2d2dbdec5657d99495a1
Sep 07 09:13:00.907894 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.907853    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"]
Sep 07 09:13:00.910016 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:13:00.909991    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32a3f9a_748d_4663_865a_d9ea2237192a.slice/crio-d0b96aa1316f6a219a1e004f7d0ad6ed20d0bd32b7ac82be493bb8fb4bb45dd3 WatchSource:0}: Error finding container d0b96aa1316f6a219a1e004f7d0ad6ed20d0bd32b7ac82be493bb8fb4bb45dd3: Status 404 returned error can't find the container with id d0b96aa1316f6a219a1e004f7d0ad6ed20d0bd32b7ac82be493bb8fb4bb45dd3
Sep 07 09:13:00.937181 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:00.937156    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"]
Sep 07 09:13:00.946629 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:13:00.946601    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f28cb92_730b_4e70_967e_d94c96e6ca87.slice/crio-296cc6b539f9edb97154a92899652666c21c93507b28a47c3a42531becc412ac WatchSource:0}: Error finding container 296cc6b539f9edb97154a92899652666c21c93507b28a47c3a42531becc412ac: Status 404 returned error can't find the container with id 296cc6b539f9edb97154a92899652666c21c93507b28a47c3a42531becc412ac
Sep 07 09:13:01.318724 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.318678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn" event={"ID":"4f28cb92-730b-4e70-967e-d94c96e6ca87","Type":"ContainerStarted","Data":"a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490"}
Sep 07 09:13:01.318926 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.318843    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:01.318926 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.318888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn" event={"ID":"4f28cb92-730b-4e70-967e-d94c96e6ca87","Type":"ContainerStarted","Data":"296cc6b539f9edb97154a92899652666c21c93507b28a47c3a42531becc412ac"}
Sep 07 09:13:01.320635 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.320605    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c" event={"ID":"b32a3f9a-748d-4663-865a-d9ea2237192a","Type":"ContainerStarted","Data":"f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448"}
Sep 07 09:13:01.320771 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.320652    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c" event={"ID":"b32a3f9a-748d-4663-865a-d9ea2237192a","Type":"ContainerStarted","Data":"d0b96aa1316f6a219a1e004f7d0ad6ed20d0bd32b7ac82be493bb8fb4bb45dd3"}
Sep 07 09:13:01.320771 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.320699    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:01.322196 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.322172    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz" event={"ID":"4db4cfcd-eaab-4bf4-be1b-c9c79daad990","Type":"ContainerStarted","Data":"a37e651907fd919cc3b7f8639c9256ec28fba23c1e0b2d2dbdec5657d99495a1"}
Sep 07 09:13:01.336617 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.336561    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn" podStartSLOduration=1.336546729 podStartE2EDuration="1.336546729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:01.334765329 +0000 UTC m=+839.303935935" watchObservedRunningTime="2026-09-07 09:13:01.336546729 +0000 UTC m=+839.305717328"
Sep 07 09:13:01.352629 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.352580    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c" podStartSLOduration=1.352564163 podStartE2EDuration="1.352564163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:01.350325235 +0000 UTC m=+839.319495835" watchObservedRunningTime="2026-09-07 09:13:01.352564163 +0000 UTC m=+839.321734789"
Sep 07 09:13:01.821457 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.821423    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"]
Sep 07 09:13:01.825028 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.824956    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:01.827746 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.827720    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-da4d4c-dockercfg-z4bsq\""
Sep 07 09:13:01.827895 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.827723    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-da4d4c\""
Sep 07 09:13:01.836189 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.836163    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"]
Sep 07 09:13:01.916199 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.916163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1497cacf-a480-4296-b250-afb66ba5d1ed-plugins-config-volume\") pod \"payload-processing-e2e-autores-da4d4c-57b8598845-p27v6\" (UID: \"1497cacf-a480-4296-b250-afb66ba5d1ed\") " pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:01.916403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:01.916209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx52x\" (UniqueName: \"kubernetes.io/projected/1497cacf-a480-4296-b250-afb66ba5d1ed-kube-api-access-wx52x\") pod \"payload-processing-e2e-autores-da4d4c-57b8598845-p27v6\" (UID: \"1497cacf-a480-4296-b250-afb66ba5d1ed\") " pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:02.017367 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.017317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1497cacf-a480-4296-b250-afb66ba5d1ed-plugins-config-volume\") pod \"payload-processing-e2e-autores-da4d4c-57b8598845-p27v6\" (UID: \"1497cacf-a480-4296-b250-afb66ba5d1ed\") " pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:02.017552 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.017395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx52x\" (UniqueName: \"kubernetes.io/projected/1497cacf-a480-4296-b250-afb66ba5d1ed-kube-api-access-wx52x\") pod \"payload-processing-e2e-autores-da4d4c-57b8598845-p27v6\" (UID: \"1497cacf-a480-4296-b250-afb66ba5d1ed\") " pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:02.018066 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.018045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1497cacf-a480-4296-b250-afb66ba5d1ed-plugins-config-volume\") pod \"payload-processing-e2e-autores-da4d4c-57b8598845-p27v6\" (UID: \"1497cacf-a480-4296-b250-afb66ba5d1ed\") " pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:02.025749 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.025715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx52x\" (UniqueName: \"kubernetes.io/projected/1497cacf-a480-4296-b250-afb66ba5d1ed-kube-api-access-wx52x\") pod \"payload-processing-e2e-autores-da4d4c-57b8598845-p27v6\" (UID: \"1497cacf-a480-4296-b250-afb66ba5d1ed\") " pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:02.137624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.137523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:02.301928 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.301899    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:13:02.328849 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.328804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz" event={"ID":"4db4cfcd-eaab-4bf4-be1b-c9c79daad990","Type":"ContainerStarted","Data":"03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6"}
Sep 07 09:13:02.329267 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.329229    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:13:02.381355 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.370785    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz" podStartSLOduration=2.839244099 podStartE2EDuration="3.370758187s" totalImagesPullingTime="531.514088ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:02.35956527 +0000 UTC m=+840.328735875" watchObservedRunningTime="2026-09-07 09:13:02.370758187 +0000 UTC m=+840.339928794"
Sep 07 09:13:02.434254 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:02.434227    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"]
Sep 07 09:13:02.436089 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:13:02.436052    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1497cacf_a480_4296_b250_afb66ba5d1ed.slice/crio-c4048c0c1aaf23e7389a501752312dd4bdc6be86a7cce131ca4bfbb169b68fe0 WatchSource:0}: Error finding container c4048c0c1aaf23e7389a501752312dd4bdc6be86a7cce131ca4bfbb169b68fe0: Status 404 returned error can't find the container with id c4048c0c1aaf23e7389a501752312dd4bdc6be86a7cce131ca4bfbb169b68fe0
Sep 07 09:13:03.337370 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:03.337332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6" event={"ID":"1497cacf-a480-4296-b250-afb66ba5d1ed","Type":"ContainerStarted","Data":"74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f"}
Sep 07 09:13:03.337849 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:03.337404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6" event={"ID":"1497cacf-a480-4296-b250-afb66ba5d1ed","Type":"ContainerStarted","Data":"c4048c0c1aaf23e7389a501752312dd4bdc6be86a7cce131ca4bfbb169b68fe0"}
Sep 07 09:13:03.337849 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:03.337590    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:03.353960 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:03.353887    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6" podStartSLOduration=2.353860837 podStartE2EDuration="2.353860837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:03.352262839 +0000 UTC m=+841.321433450" watchObservedRunningTime="2026-09-07 09:13:03.353860837 +0000 UTC m=+841.323031443"
Sep 07 09:13:08.303804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:08.303771    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:13:09.306495 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:09.306440    2563 patch_prober.go:28] interesting pod/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 09:13:09.307011 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:09.306505    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 09:13:09.357262 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:09.357231    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:13:11.404562 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:11.404522    2563 generic.go:356] "Generic (PLEG): container finished" podID="26a709fe-0870-4529-905c-5075b7758948" containerID="29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084" exitCode=0
Sep 07 09:13:11.404982 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:11.404596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" event={"ID":"26a709fe-0870-4529-905c-5075b7758948","Type":"ContainerDied","Data":"29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084"}
Sep 07 09:13:11.405029 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:11.405001    2563 scope.go:117] "RemoveContainer" containerID="29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084"
Sep 07 09:13:11.406053 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:11.406018    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:13:11.406148 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:11.406085    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:13:11.406148 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:11.406119    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:13:12.331033 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:12.331005    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:12.331243 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:12.331087    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:12.412954 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:12.412912    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" event={"ID":"26a709fe-0870-4529-905c-5075b7758948","Type":"ContainerStarted","Data":"085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba"}
Sep 07 09:13:12.930357 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:12.930324    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:13:12.930602 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:12.930579    2563 patch_prober.go:28] interesting pod/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 09:13:12.930660 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:12.930640    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 09:13:13.930594 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:13.930561    2563 patch_prober.go:28] interesting pod/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 09:13:13.931069 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:13.931041    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 09:13:14.354480 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:14.354442    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:13:14.929831 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:14.929790    2563 patch_prober.go:28] interesting pod/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 09:13:14.930014 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:14.929864    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 09:13:15.930491 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:15.930447    2563 patch_prober.go:28] interesting pod/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 09:13:15.930898 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:15.930520    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 09:13:16.930468 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:16.930431    2563 patch_prober.go:28] interesting pod/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 09:13:16.930639 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:16.930485    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 09:13:17.933455 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:17.933417    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:13:17.933893 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:17.933702    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:13:17.934495 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:17.934474    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:13:24.219432 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.219382    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"]
Sep 07 09:13:24.222185 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.222167    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:24.224706 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.224680    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-3bcf58\""
Sep 07 09:13:24.224706 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.224693    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-3bcf58-dockercfg-2sm9c\""
Sep 07 09:13:24.233428 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.233401    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"]
Sep 07 09:13:24.258264 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.258231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8\" (UID: \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:24.258456 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.258352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv4dh\" (UniqueName: \"kubernetes.io/projected/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-kube-api-access-sv4dh\") pod \"payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8\" (UID: \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:24.359707 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.359672    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8\" (UID: \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:24.359909 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.359737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv4dh\" (UniqueName: \"kubernetes.io/projected/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-kube-api-access-sv4dh\") pod \"payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8\" (UID: \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:24.360419 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.360393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8\" (UID: \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:24.367534 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.367506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv4dh\" (UniqueName: \"kubernetes.io/projected/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-kube-api-access-sv4dh\") pod \"payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8\" (UID: \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:24.534481 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.534444    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:24.811389 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:24.811361    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"]
Sep 07 09:13:24.813365 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:13:24.813275    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd29d14_a61d_4e3f_9fd9_db45a4a6f0b2.slice/crio-f6127f97bc9835b30d80e892e8ca1d336a9699f0d39994beefd7a044f7abfb79 WatchSource:0}: Error finding container f6127f97bc9835b30d80e892e8ca1d336a9699f0d39994beefd7a044f7abfb79: Status 404 returned error can't find the container with id f6127f97bc9835b30d80e892e8ca1d336a9699f0d39994beefd7a044f7abfb79
Sep 07 09:13:25.486647 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:25.486603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8" event={"ID":"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2","Type":"ContainerStarted","Data":"caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87"}
Sep 07 09:13:25.487151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:25.486698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8" event={"ID":"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2","Type":"ContainerStarted","Data":"f6127f97bc9835b30d80e892e8ca1d336a9699f0d39994beefd7a044f7abfb79"}
Sep 07 09:13:25.487151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:25.486729    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:25.505834 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:25.505774    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8" podStartSLOduration=1.505757339 podStartE2EDuration="1.505757339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:25.505235788 +0000 UTC m=+863.474406393" watchObservedRunningTime="2026-09-07 09:13:25.505757339 +0000 UTC m=+863.474927957"
Sep 07 09:13:36.495450 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:36.495420    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:13:40.510217 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.510177    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"]
Sep 07 09:13:40.510696 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.510467    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz" podUID="4db4cfcd-eaab-4bf4-be1b-c9c79daad990" containerName="maas-api" containerID="cri-o://03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6" gracePeriod=30
Sep 07 09:13:40.572132 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.572102    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"]
Sep 07 09:13:40.572342 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.572315    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn" podUID="4f28cb92-730b-4e70-967e-d94c96e6ca87" containerName="payload-processing" containerID="cri-o://a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490" gracePeriod=30
Sep 07 09:13:40.579729 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.579538    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"]
Sep 07 09:13:40.579862 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.579763    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c" podUID="b32a3f9a-748d-4663-865a-d9ea2237192a" containerName="payload-pre-processing" containerID="cri-o://f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448" gracePeriod=30
Sep 07 09:13:40.854108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.854074    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:13:40.885216 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.885185    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:40.893732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.893701    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:40.914698 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.914669    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmmn7\" (UniqueName: \"kubernetes.io/projected/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-kube-api-access-hmmn7\") pod \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") "
Sep 07 09:13:40.914698 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.914702    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-openshift-service-ca\") pod \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") "
Sep 07 09:13:40.914949 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.914728    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-metrics-certs\") pod \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") "
Sep 07 09:13:40.914949 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.914818    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-maas-api-tls\") pod \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\" (UID: \"4db4cfcd-eaab-4bf4-be1b-c9c79daad990\") "
Sep 07 09:13:40.915128 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.915104    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4db4cfcd-eaab-4bf4-be1b-c9c79daad990" (UID: "4db4cfcd-eaab-4bf4-be1b-c9c79daad990"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:13:40.917662 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.917627    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-kube-api-access-hmmn7" (OuterVolumeSpecName: "kube-api-access-hmmn7") pod "4db4cfcd-eaab-4bf4-be1b-c9c79daad990" (UID: "4db4cfcd-eaab-4bf4-be1b-c9c79daad990"). InnerVolumeSpecName "kube-api-access-hmmn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:13:40.917793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.917669    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4db4cfcd-eaab-4bf4-be1b-c9c79daad990" (UID: "4db4cfcd-eaab-4bf4-be1b-c9c79daad990"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:13:40.917793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:40.917691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4db4cfcd-eaab-4bf4-be1b-c9c79daad990" (UID: "4db4cfcd-eaab-4bf4-be1b-c9c79daad990"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:13:41.015908 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.015870    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b32a3f9a-748d-4663-865a-d9ea2237192a-plugins-config-volume\") pod \"b32a3f9a-748d-4663-865a-d9ea2237192a\" (UID: \"b32a3f9a-748d-4663-865a-d9ea2237192a\") "
Sep 07 09:13:41.016113 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.015949    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f28cb92-730b-4e70-967e-d94c96e6ca87-plugins-config-volume\") pod \"4f28cb92-730b-4e70-967e-d94c96e6ca87\" (UID: \"4f28cb92-730b-4e70-967e-d94c96e6ca87\") "
Sep 07 09:13:41.016113 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.015998    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znf7h\" (UniqueName: \"kubernetes.io/projected/b32a3f9a-748d-4663-865a-d9ea2237192a-kube-api-access-znf7h\") pod \"b32a3f9a-748d-4663-865a-d9ea2237192a\" (UID: \"b32a3f9a-748d-4663-865a-d9ea2237192a\") "
Sep 07 09:13:41.016113 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.016032    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kztr2\" (UniqueName: \"kubernetes.io/projected/4f28cb92-730b-4e70-967e-d94c96e6ca87-kube-api-access-kztr2\") pod \"4f28cb92-730b-4e70-967e-d94c96e6ca87\" (UID: \"4f28cb92-730b-4e70-967e-d94c96e6ca87\") "
Sep 07 09:13:41.016305 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.016219    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:13:41.016305 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.016236    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmmn7\" (UniqueName: \"kubernetes.io/projected/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-kube-api-access-hmmn7\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:13:41.016305 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.016253    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:13:41.016305 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.016267    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4db4cfcd-eaab-4bf4-be1b-c9c79daad990-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:13:41.016482 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.016308    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b32a3f9a-748d-4663-865a-d9ea2237192a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b32a3f9a-748d-4663-865a-d9ea2237192a" (UID: "b32a3f9a-748d-4663-865a-d9ea2237192a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:13:41.016482 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.016346    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f28cb92-730b-4e70-967e-d94c96e6ca87-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f28cb92-730b-4e70-967e-d94c96e6ca87" (UID: "4f28cb92-730b-4e70-967e-d94c96e6ca87"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:13:41.018227 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.018207    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f28cb92-730b-4e70-967e-d94c96e6ca87-kube-api-access-kztr2" (OuterVolumeSpecName: "kube-api-access-kztr2") pod "4f28cb92-730b-4e70-967e-d94c96e6ca87" (UID: "4f28cb92-730b-4e70-967e-d94c96e6ca87"). InnerVolumeSpecName "kube-api-access-kztr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:13:41.018329 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.018250    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b32a3f9a-748d-4663-865a-d9ea2237192a-kube-api-access-znf7h" (OuterVolumeSpecName: "kube-api-access-znf7h") pod "b32a3f9a-748d-4663-865a-d9ea2237192a" (UID: "b32a3f9a-748d-4663-865a-d9ea2237192a"). InnerVolumeSpecName "kube-api-access-znf7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:13:41.116913 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.116872    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f28cb92-730b-4e70-967e-d94c96e6ca87-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:13:41.116913 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.116906    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znf7h\" (UniqueName: \"kubernetes.io/projected/b32a3f9a-748d-4663-865a-d9ea2237192a-kube-api-access-znf7h\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:13:41.116913 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.116919    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kztr2\" (UniqueName: \"kubernetes.io/projected/4f28cb92-730b-4e70-967e-d94c96e6ca87-kube-api-access-kztr2\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:13:41.117140 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.116933    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b32a3f9a-748d-4663-865a-d9ea2237192a-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:13:41.574209 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.574174    2563 generic.go:356] "Generic (PLEG): container finished" podID="4f28cb92-730b-4e70-967e-d94c96e6ca87" containerID="a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490" exitCode=0
Sep 07 09:13:41.574690 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.574218    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"
Sep 07 09:13:41.574690 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.574261    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn" event={"ID":"4f28cb92-730b-4e70-967e-d94c96e6ca87","Type":"ContainerDied","Data":"a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490"}
Sep 07 09:13:41.574690 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.574346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn" event={"ID":"4f28cb92-730b-4e70-967e-d94c96e6ca87","Type":"ContainerDied","Data":"296cc6b539f9edb97154a92899652666c21c93507b28a47c3a42531becc412ac"}
Sep 07 09:13:41.574690 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.574383    2563 scope.go:117] "RemoveContainer" containerID="a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490"
Sep 07 09:13:41.575754 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.575716    2563 generic.go:356] "Generic (PLEG): container finished" podID="b32a3f9a-748d-4663-865a-d9ea2237192a" containerID="f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448" exitCode=0
Sep 07 09:13:41.575830 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.575747    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c" event={"ID":"b32a3f9a-748d-4663-865a-d9ea2237192a","Type":"ContainerDied","Data":"f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448"}
Sep 07 09:13:41.575830 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.575771    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"
Sep 07 09:13:41.575830 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.575799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c" event={"ID":"b32a3f9a-748d-4663-865a-d9ea2237192a","Type":"ContainerDied","Data":"d0b96aa1316f6a219a1e004f7d0ad6ed20d0bd32b7ac82be493bb8fb4bb45dd3"}
Sep 07 09:13:41.577470 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.577368    2563 generic.go:356] "Generic (PLEG): container finished" podID="4db4cfcd-eaab-4bf4-be1b-c9c79daad990" containerID="03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6" exitCode=0
Sep 07 09:13:41.577470 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.577411    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"
Sep 07 09:13:41.577470 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.577421    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz" event={"ID":"4db4cfcd-eaab-4bf4-be1b-c9c79daad990","Type":"ContainerDied","Data":"03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6"}
Sep 07 09:13:41.577470 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.577461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz" event={"ID":"4db4cfcd-eaab-4bf4-be1b-c9c79daad990","Type":"ContainerDied","Data":"a37e651907fd919cc3b7f8639c9256ec28fba23c1e0b2d2dbdec5657d99495a1"}
Sep 07 09:13:41.584353 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.584328    2563 scope.go:117] "RemoveContainer" containerID="a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490"
Sep 07 09:13:41.584735 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:13:41.584708    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490\": container with ID starting with a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490 not found: ID does not exist" containerID="a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490"
Sep 07 09:13:41.584855 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.584747    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490"} err="failed to get container status \"a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490\": rpc error: code = NotFound desc = could not find container \"a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490\": container with ID starting with a88dd045b06fe28b2d21907ca993f8609a39749b483d63c18b782257575a2490 not found: ID does not exist"
Sep 07 09:13:41.584855 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.584772    2563 scope.go:117] "RemoveContainer" containerID="f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448"
Sep 07 09:13:41.593653 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.593632    2563 scope.go:117] "RemoveContainer" containerID="f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448"
Sep 07 09:13:41.593977 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:13:41.593960    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448\": container with ID starting with f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448 not found: ID does not exist" containerID="f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448"
Sep 07 09:13:41.594052 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.593983    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448"} err="failed to get container status \"f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448\": rpc error: code = NotFound desc = could not find container \"f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448\": container with ID starting with f74280ed85add78780d72d68ab53ea63f52f363c5a43b9f989ed9f5ae9d40448 not found: ID does not exist"
Sep 07 09:13:41.594052 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.593999    2563 scope.go:117] "RemoveContainer" containerID="03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6"
Sep 07 09:13:41.602476 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.602454    2563 scope.go:117] "RemoveContainer" containerID="03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6"
Sep 07 09:13:41.602803 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:13:41.602783    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6\": container with ID starting with 03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6 not found: ID does not exist" containerID="03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6"
Sep 07 09:13:41.602865 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.602814    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6"} err="failed to get container status \"03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6\": rpc error: code = NotFound desc = could not find container \"03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6\": container with ID starting with 03e9704b85c9b96696b9744db977c999925b54639c3dc038e40307c7b9dcd2b6 not found: ID does not exist"
Sep 07 09:13:41.603774 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.603753    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"]
Sep 07 09:13:41.609603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.609574    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-140be7ab-747d4fbd47-kc7vn"]
Sep 07 09:13:41.620221 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.620192    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"]
Sep 07 09:13:41.624514 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.624487    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-140be7ab-bd5dcfb5d-sxvnz"]
Sep 07 09:13:41.635914 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.635871    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"]
Sep 07 09:13:41.641856 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.641828    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-140be7ab-567fff8446-sfq9c"]
Sep 07 09:13:41.759363 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759333    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"]
Sep 07 09:13:41.759726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759714    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b32a3f9a-748d-4663-865a-d9ea2237192a" containerName="payload-pre-processing"
Sep 07 09:13:41.759766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759727    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b32a3f9a-748d-4663-865a-d9ea2237192a" containerName="payload-pre-processing"
Sep 07 09:13:41.759766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759737    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f28cb92-730b-4e70-967e-d94c96e6ca87" containerName="payload-processing"
Sep 07 09:13:41.759766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759744    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f28cb92-730b-4e70-967e-d94c96e6ca87" containerName="payload-processing"
Sep 07 09:13:41.759766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759753    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4db4cfcd-eaab-4bf4-be1b-c9c79daad990" containerName="maas-api"
Sep 07 09:13:41.759766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759759    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4db4cfcd-eaab-4bf4-be1b-c9c79daad990" containerName="maas-api"
Sep 07 09:13:41.759926 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759816    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b32a3f9a-748d-4663-865a-d9ea2237192a" containerName="payload-pre-processing"
Sep 07 09:13:41.759926 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759825    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4db4cfcd-eaab-4bf4-be1b-c9c79daad990" containerName="maas-api"
Sep 07 09:13:41.759926 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.759831    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f28cb92-730b-4e70-967e-d94c96e6ca87" containerName="payload-processing"
Sep 07 09:13:41.764111 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.764091    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.767008 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.766984    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-7928b616-gw-openshift-default-dockercfg-ppmqm\""
Sep 07 09:13:41.777111 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.777057    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"]
Sep 07 09:13:41.824112 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.824079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-podinfo\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.824349 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.824119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-envoy\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.824349 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.824137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-token\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.824349 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.824157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-credential-socket\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.824349 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.824265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-data\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.824520 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.824411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-socket\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.824520 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.824437    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19189c89-4e3f-4153-8ad8-5c29aee3848f-istiod-ca-cert\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.824520 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.824457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzknl\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-kube-api-access-nzknl\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.824520 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.824489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-certs\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.925546 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.925513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-socket\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.925726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.925558    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19189c89-4e3f-4153-8ad8-5c29aee3848f-istiod-ca-cert\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.925726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.925579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzknl\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-kube-api-access-nzknl\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.925726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.925603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-certs\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.925726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.925631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-podinfo\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.925726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.925666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-envoy\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.925726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.925688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-token\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.925989 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.925717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-credential-socket\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.925989 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.925789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-data\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.926597 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.926338    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-socket\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.926731 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.926607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-certs\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.927366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.926817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-data\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.927366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.926893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19189c89-4e3f-4153-8ad8-5c29aee3848f-istiod-ca-cert\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.927366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.927055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-credential-socket\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.933885 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.933857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-envoy\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.934432 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.934411    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-podinfo\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.936332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.936312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-token\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:41.936463 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:41.936443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzknl\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-kube-api-access-nzknl\") pod \"e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") " pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:42.075569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.075474    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:42.375154 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.375126    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"]
Sep 07 09:13:42.376698 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:13:42.376673    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19189c89_4e3f_4153_8ad8_5c29aee3848f.slice/crio-c38df0b2a0e0e02bd96ca5c88b7ea8e344e7b6557df4c432545104e6d0937aaa WatchSource:0}: Error finding container c38df0b2a0e0e02bd96ca5c88b7ea8e344e7b6557df4c432545104e6d0937aaa: Status 404 returned error can't find the container with id c38df0b2a0e0e02bd96ca5c88b7ea8e344e7b6557df4c432545104e6d0937aaa
Sep 07 09:13:42.378822 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.378786    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:13:42.378892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.378857    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:13:42.378892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.378884    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:13:42.465828 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.465790    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4db4cfcd-eaab-4bf4-be1b-c9c79daad990" path="/var/lib/kubelet/pods/4db4cfcd-eaab-4bf4-be1b-c9c79daad990/volumes"
Sep 07 09:13:42.466461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.466437    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f28cb92-730b-4e70-967e-d94c96e6ca87" path="/var/lib/kubelet/pods/4f28cb92-730b-4e70-967e-d94c96e6ca87/volumes"
Sep 07 09:13:42.466936 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.466911    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b32a3f9a-748d-4663-865a-d9ea2237192a" path="/var/lib/kubelet/pods/b32a3f9a-748d-4663-865a-d9ea2237192a/volumes"
Sep 07 09:13:42.587002 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.586964    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" event={"ID":"19189c89-4e3f-4153-8ad8-5c29aee3848f","Type":"ContainerStarted","Data":"18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107"}
Sep 07 09:13:42.587483 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.587024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" event={"ID":"19189c89-4e3f-4153-8ad8-5c29aee3848f","Type":"ContainerStarted","Data":"c38df0b2a0e0e02bd96ca5c88b7ea8e344e7b6557df4c432545104e6d0937aaa"}
Sep 07 09:13:42.608516 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:42.608455    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" podStartSLOduration=1.608434337 podStartE2EDuration="1.608434337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:42.605275506 +0000 UTC m=+880.574446126" watchObservedRunningTime="2026-09-07 09:13:42.608434337 +0000 UTC m=+880.577604945"
Sep 07 09:13:43.076204 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:43.076166    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:43.081197 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:43.081169    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:43.592793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:43.592761    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:43.593747 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:43.593727    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:13:44.300069 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.300034    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"]
Sep 07 09:13:44.304740 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.304710    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.307347 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.307319    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-7928b616-serving-cert\""
Sep 07 09:13:44.317494 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.317452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"]
Sep 07 09:13:44.450009 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.449968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v67fw\" (UniqueName: \"kubernetes.io/projected/8137ccb6-4348-4057-b7af-66bb5ed3d02f-kube-api-access-v67fw\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.450009 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.450011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-maas-api-tls\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.450275 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.450038    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8137ccb6-4348-4057-b7af-66bb5ed3d02f-openshift-service-ca\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.450275 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.450152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-metrics-certs\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.551732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.551638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v67fw\" (UniqueName: \"kubernetes.io/projected/8137ccb6-4348-4057-b7af-66bb5ed3d02f-kube-api-access-v67fw\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.551939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.551811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-maas-api-tls\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.551939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.551859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8137ccb6-4348-4057-b7af-66bb5ed3d02f-openshift-service-ca\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.552055 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.551967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-metrics-certs\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.552539 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.552520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8137ccb6-4348-4057-b7af-66bb5ed3d02f-openshift-service-ca\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.554322 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.554303    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-metrics-certs\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.554380 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.554358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-maas-api-tls\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.562469 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.562445    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v67fw\" (UniqueName: \"kubernetes.io/projected/8137ccb6-4348-4057-b7af-66bb5ed3d02f-kube-api-access-v67fw\") pod \"maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.618684 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.618644    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:44.901548 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:13:44.901520    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8137ccb6_4348_4057_b7af_66bb5ed3d02f.slice/crio-e6a9ebe0970c32945072700dd0c610647b6807dbf19218fc98bee104ba9bb6bd WatchSource:0}: Error finding container e6a9ebe0970c32945072700dd0c610647b6807dbf19218fc98bee104ba9bb6bd: Status 404 returned error can't find the container with id e6a9ebe0970c32945072700dd0c610647b6807dbf19218fc98bee104ba9bb6bd
Sep 07 09:13:44.918446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.918417    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"]
Sep 07 09:13:44.969707 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.969677    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"]
Sep 07 09:13:44.974740 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.974718    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:13:44.977312 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.977275    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-7928b616-dockercfg-z5sjz\""
Sep 07 09:13:44.977399 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.977339    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-7928b616\""
Sep 07 09:13:44.984430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:44.984403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"]
Sep 07 09:13:45.004033 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.003998    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"]
Sep 07 09:13:45.007524 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.007506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:45.016892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.016865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"]
Sep 07 09:13:45.056599 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.056558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd66c31-3562-4381-bdbd-aaaf2a76d679-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7\" (UID: \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:13:45.056747 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.056628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6rzw\" (UniqueName: \"kubernetes.io/projected/cdd66c31-3562-4381-bdbd-aaaf2a76d679-kube-api-access-d6rzw\") pod \"payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7\" (UID: \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:13:45.157460 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.157374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd66c31-3562-4381-bdbd-aaaf2a76d679-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7\" (UID: \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:13:45.157460 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.157417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d535cf66-3380-43c0-bd7d-2fded82ff47f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s\" (UID: \"d535cf66-3380-43c0-bd7d-2fded82ff47f\") " pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:45.157653 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.157469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6rzw\" (UniqueName: \"kubernetes.io/projected/cdd66c31-3562-4381-bdbd-aaaf2a76d679-kube-api-access-d6rzw\") pod \"payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7\" (UID: \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:13:45.157653 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.157495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5kgg\" (UniqueName: \"kubernetes.io/projected/d535cf66-3380-43c0-bd7d-2fded82ff47f-kube-api-access-m5kgg\") pod \"payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s\" (UID: \"d535cf66-3380-43c0-bd7d-2fded82ff47f\") " pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:45.158112 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.158091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd66c31-3562-4381-bdbd-aaaf2a76d679-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7\" (UID: \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:13:45.164807 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.164776    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6rzw\" (UniqueName: \"kubernetes.io/projected/cdd66c31-3562-4381-bdbd-aaaf2a76d679-kube-api-access-d6rzw\") pod \"payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7\" (UID: \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:13:45.258354 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.258324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5kgg\" (UniqueName: \"kubernetes.io/projected/d535cf66-3380-43c0-bd7d-2fded82ff47f-kube-api-access-m5kgg\") pod \"payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s\" (UID: \"d535cf66-3380-43c0-bd7d-2fded82ff47f\") " pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:45.258485 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.258465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d535cf66-3380-43c0-bd7d-2fded82ff47f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s\" (UID: \"d535cf66-3380-43c0-bd7d-2fded82ff47f\") " pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:45.259087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.259067    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d535cf66-3380-43c0-bd7d-2fded82ff47f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s\" (UID: \"d535cf66-3380-43c0-bd7d-2fded82ff47f\") " pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:45.266898 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.266877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5kgg\" (UniqueName: \"kubernetes.io/projected/d535cf66-3380-43c0-bd7d-2fded82ff47f-kube-api-access-m5kgg\") pod \"payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s\" (UID: \"d535cf66-3380-43c0-bd7d-2fded82ff47f\") " pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:45.287502 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.287478    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:13:45.320821 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.320786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:45.559946 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.559917    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"]
Sep 07 09:13:45.561748 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:13:45.561723    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd66c31_3562_4381_bdbd_aaaf2a76d679.slice/crio-d292ba38fd1fa645f0dad0642e5206d592333bf37fe2f879965cd0b37ec86f11 WatchSource:0}: Error finding container d292ba38fd1fa645f0dad0642e5206d592333bf37fe2f879965cd0b37ec86f11: Status 404 returned error can't find the container with id d292ba38fd1fa645f0dad0642e5206d592333bf37fe2f879965cd0b37ec86f11
Sep 07 09:13:45.604591 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.604560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7" event={"ID":"cdd66c31-3562-4381-bdbd-aaaf2a76d679","Type":"ContainerStarted","Data":"d292ba38fd1fa645f0dad0642e5206d592333bf37fe2f879965cd0b37ec86f11"}
Sep 07 09:13:45.607187 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.607141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2" event={"ID":"8137ccb6-4348-4057-b7af-66bb5ed3d02f","Type":"ContainerStarted","Data":"79d72e868e60a80d29a07db4ce996c2f3ae0bca55b8f08ef333c959d4a1f7d7d"}
Sep 07 09:13:45.607355 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.607197    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2" event={"ID":"8137ccb6-4348-4057-b7af-66bb5ed3d02f","Type":"ContainerStarted","Data":"e6a9ebe0970c32945072700dd0c610647b6807dbf19218fc98bee104ba9bb6bd"}
Sep 07 09:13:45.607355 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.607302    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:45.609214 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.609192    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"]
Sep 07 09:13:45.622690 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:45.622637    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2" podStartSLOduration=1.293714852 podStartE2EDuration="1.622624227s" totalImagesPullingTime="328.909375ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:45.622167249 +0000 UTC m=+883.591337854" watchObservedRunningTime="2026-09-07 09:13:45.622624227 +0000 UTC m=+883.591794831"
Sep 07 09:13:45.624415 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:13:45.624389    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd535cf66_3380_43c0_bd7d_2fded82ff47f.slice/crio-f2c8333a7ecf0fe186d4f4c729e465cd50ffe5b93cda759bb2493efd35a81c9d WatchSource:0}: Error finding container f2c8333a7ecf0fe186d4f4c729e465cd50ffe5b93cda759bb2493efd35a81c9d: Status 404 returned error can't find the container with id f2c8333a7ecf0fe186d4f4c729e465cd50ffe5b93cda759bb2493efd35a81c9d
Sep 07 09:13:46.613049 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:46.613009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s" event={"ID":"d535cf66-3380-43c0-bd7d-2fded82ff47f","Type":"ContainerStarted","Data":"e4c51c0dd05e1f188bda6120941ed8409d6ebb55dd53cf20247ca07a73c9d1e0"}
Sep 07 09:13:46.613388 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:46.613083    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:46.613388 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:46.613110    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s" event={"ID":"d535cf66-3380-43c0-bd7d-2fded82ff47f","Type":"ContainerStarted","Data":"f2c8333a7ecf0fe186d4f4c729e465cd50ffe5b93cda759bb2493efd35a81c9d"}
Sep 07 09:13:46.614561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:46.614526    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7" event={"ID":"cdd66c31-3562-4381-bdbd-aaaf2a76d679","Type":"ContainerStarted","Data":"0c3d13c7333247f87d42d60ce0eaa933a204b9ee7ed2534c9ab390cfe340dd4e"}
Sep 07 09:13:46.614726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:46.614710    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:13:46.628327 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:46.628251    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s" podStartSLOduration=2.628237406 podStartE2EDuration="2.628237406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:46.627402523 +0000 UTC m=+884.596573129" watchObservedRunningTime="2026-09-07 09:13:46.628237406 +0000 UTC m=+884.597408055"
Sep 07 09:13:46.644793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:46.644734    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7" podStartSLOduration=2.644716779 podStartE2EDuration="2.644716779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:46.642411163 +0000 UTC m=+884.611581767" watchObservedRunningTime="2026-09-07 09:13:46.644716779 +0000 UTC m=+884.613887385"
Sep 07 09:13:51.620627 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:51.620595    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:13:57.622275 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:57.622197    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:13:57.622275 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:13:57.622269    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:14:04.725510 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:04.725473    2563 generic.go:356] "Generic (PLEG): container finished" podID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerID="18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107" exitCode=0
Sep 07 09:14:04.725933 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:04.725544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" event={"ID":"19189c89-4e3f-4153-8ad8-5c29aee3848f","Type":"ContainerDied","Data":"18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107"}
Sep 07 09:14:04.725978 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:04.725962    2563 scope.go:117] "RemoveContainer" containerID="18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107"
Sep 07 09:14:04.727104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:04.727071    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:04.727204 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:04.727158    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:04.727268 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:04.727209    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:05.741478 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:05.741435    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" event={"ID":"19189c89-4e3f-4153-8ad8-5c29aee3848f","Type":"ContainerStarted","Data":"3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2"}
Sep 07 09:14:06.076448 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:06.076414    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:14:06.076628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:06.076522    2563 patch_prober.go:28] interesting pod/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 09:14:06.076628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:06.076561    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 09:14:07.076202 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:07.076161    2563 patch_prober.go:28] interesting pod/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 09:14:07.076667 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:07.076227    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 09:14:08.076584 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:08.076544    2563 patch_prober.go:28] interesting pod/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 09:14:08.076987 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:08.076601    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 09:14:09.076066 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:09.076020    2563 patch_prober.go:28] interesting pod/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 09:14:09.076319 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:09.076094    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 09:14:10.075976 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:10.075936    2563 patch_prober.go:28] interesting pod/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 09:14:10.076406 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:10.076031    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 09:14:11.076779 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:11.076736    2563 patch_prober.go:28] interesting pod/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 09:14:11.077142 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:11.076810    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 09:14:12.081338 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:12.081304    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:14:12.081845 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:12.081823    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:14:12.082461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:12.082440    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:14:15.580505 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.580465    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"]
Sep 07 09:14:15.580990 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.580682    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2" podUID="8137ccb6-4348-4057-b7af-66bb5ed3d02f" containerName="maas-api" containerID="cri-o://79d72e868e60a80d29a07db4ce996c2f3ae0bca55b8f08ef333c959d4a1f7d7d" gracePeriod=30
Sep 07 09:14:15.632149 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.632114    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"]
Sep 07 09:14:15.632403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.632337    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s" podUID="d535cf66-3380-43c0-bd7d-2fded82ff47f" containerName="payload-processing" containerID="cri-o://e4c51c0dd05e1f188bda6120941ed8409d6ebb55dd53cf20247ca07a73c9d1e0" gracePeriod=30
Sep 07 09:14:15.637655 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.637626    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"]
Sep 07 09:14:15.637959 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.637888    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7" podUID="cdd66c31-3562-4381-bdbd-aaaf2a76d679" containerName="payload-pre-processing" containerID="cri-o://0c3d13c7333247f87d42d60ce0eaa933a204b9ee7ed2534c9ab390cfe340dd4e" gracePeriod=30
Sep 07 09:14:15.809484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.809451    2563 generic.go:356] "Generic (PLEG): container finished" podID="d535cf66-3380-43c0-bd7d-2fded82ff47f" containerID="e4c51c0dd05e1f188bda6120941ed8409d6ebb55dd53cf20247ca07a73c9d1e0" exitCode=0
Sep 07 09:14:15.809624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.809531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s" event={"ID":"d535cf66-3380-43c0-bd7d-2fded82ff47f","Type":"ContainerDied","Data":"e4c51c0dd05e1f188bda6120941ed8409d6ebb55dd53cf20247ca07a73c9d1e0"}
Sep 07 09:14:15.811991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.811816    2563 generic.go:356] "Generic (PLEG): container finished" podID="cdd66c31-3562-4381-bdbd-aaaf2a76d679" containerID="0c3d13c7333247f87d42d60ce0eaa933a204b9ee7ed2534c9ab390cfe340dd4e" exitCode=0
Sep 07 09:14:15.811991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.811936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7" event={"ID":"cdd66c31-3562-4381-bdbd-aaaf2a76d679","Type":"ContainerDied","Data":"0c3d13c7333247f87d42d60ce0eaa933a204b9ee7ed2534c9ab390cfe340dd4e"}
Sep 07 09:14:15.814171 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.814067    2563 generic.go:356] "Generic (PLEG): container finished" podID="8137ccb6-4348-4057-b7af-66bb5ed3d02f" containerID="79d72e868e60a80d29a07db4ce996c2f3ae0bca55b8f08ef333c959d4a1f7d7d" exitCode=0
Sep 07 09:14:15.814171 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.814116    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2" event={"ID":"8137ccb6-4348-4057-b7af-66bb5ed3d02f","Type":"ContainerDied","Data":"79d72e868e60a80d29a07db4ce996c2f3ae0bca55b8f08ef333c959d4a1f7d7d"}
Sep 07 09:14:15.914952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.912174    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"]
Sep 07 09:14:15.914952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.912426    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy" containerID="cri-o://3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2" gracePeriod=30
Sep 07 09:14:15.933080 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.932813    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:14:15.967090 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.967059    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:14:15.978086 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:15.978061    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:14:16.065839 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.065801    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd66c31-3562-4381-bdbd-aaaf2a76d679-plugins-config-volume\") pod \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\" (UID: \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\") "
Sep 07 09:14:16.065839 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.065847    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6rzw\" (UniqueName: \"kubernetes.io/projected/cdd66c31-3562-4381-bdbd-aaaf2a76d679-kube-api-access-d6rzw\") pod \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\" (UID: \"cdd66c31-3562-4381-bdbd-aaaf2a76d679\") "
Sep 07 09:14:16.066070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.065874    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-maas-api-tls\") pod \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") "
Sep 07 09:14:16.066070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.065911    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8137ccb6-4348-4057-b7af-66bb5ed3d02f-openshift-service-ca\") pod \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") "
Sep 07 09:14:16.066183 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066074    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-metrics-certs\") pod \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") "
Sep 07 09:14:16.066183 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066151    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d535cf66-3380-43c0-bd7d-2fded82ff47f-plugins-config-volume\") pod \"d535cf66-3380-43c0-bd7d-2fded82ff47f\" (UID: \"d535cf66-3380-43c0-bd7d-2fded82ff47f\") "
Sep 07 09:14:16.066300 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066185    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v67fw\" (UniqueName: \"kubernetes.io/projected/8137ccb6-4348-4057-b7af-66bb5ed3d02f-kube-api-access-v67fw\") pod \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\" (UID: \"8137ccb6-4348-4057-b7af-66bb5ed3d02f\") "
Sep 07 09:14:16.066300 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066271    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5kgg\" (UniqueName: \"kubernetes.io/projected/d535cf66-3380-43c0-bd7d-2fded82ff47f-kube-api-access-m5kgg\") pod \"d535cf66-3380-43c0-bd7d-2fded82ff47f\" (UID: \"d535cf66-3380-43c0-bd7d-2fded82ff47f\") "
Sep 07 09:14:16.066407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066305    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8137ccb6-4348-4057-b7af-66bb5ed3d02f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8137ccb6-4348-4057-b7af-66bb5ed3d02f" (UID: "8137ccb6-4348-4057-b7af-66bb5ed3d02f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:14:16.066407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066350    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd66c31-3562-4381-bdbd-aaaf2a76d679-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cdd66c31-3562-4381-bdbd-aaaf2a76d679" (UID: "cdd66c31-3562-4381-bdbd-aaaf2a76d679"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:14:16.066527 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066504    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d535cf66-3380-43c0-bd7d-2fded82ff47f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d535cf66-3380-43c0-bd7d-2fded82ff47f" (UID: "d535cf66-3380-43c0-bd7d-2fded82ff47f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:14:16.066741 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066715    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd66c31-3562-4381-bdbd-aaaf2a76d679-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:16.066741 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066743    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8137ccb6-4348-4057-b7af-66bb5ed3d02f-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:16.067041 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.066759    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d535cf66-3380-43c0-bd7d-2fded82ff47f-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:16.068372 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.068343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8137ccb6-4348-4057-b7af-66bb5ed3d02f" (UID: "8137ccb6-4348-4057-b7af-66bb5ed3d02f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:14:16.068486 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.068406    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8137ccb6-4348-4057-b7af-66bb5ed3d02f-kube-api-access-v67fw" (OuterVolumeSpecName: "kube-api-access-v67fw") pod "8137ccb6-4348-4057-b7af-66bb5ed3d02f" (UID: "8137ccb6-4348-4057-b7af-66bb5ed3d02f"). InnerVolumeSpecName "kube-api-access-v67fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:16.068486 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.068440    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd66c31-3562-4381-bdbd-aaaf2a76d679-kube-api-access-d6rzw" (OuterVolumeSpecName: "kube-api-access-d6rzw") pod "cdd66c31-3562-4381-bdbd-aaaf2a76d679" (UID: "cdd66c31-3562-4381-bdbd-aaaf2a76d679"). InnerVolumeSpecName "kube-api-access-d6rzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:16.068680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.068659    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8137ccb6-4348-4057-b7af-66bb5ed3d02f" (UID: "8137ccb6-4348-4057-b7af-66bb5ed3d02f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:14:16.068726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.068684    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d535cf66-3380-43c0-bd7d-2fded82ff47f-kube-api-access-m5kgg" (OuterVolumeSpecName: "kube-api-access-m5kgg") pod "d535cf66-3380-43c0-bd7d-2fded82ff47f" (UID: "d535cf66-3380-43c0-bd7d-2fded82ff47f"). InnerVolumeSpecName "kube-api-access-m5kgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:16.168200 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.168109    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5kgg\" (UniqueName: \"kubernetes.io/projected/d535cf66-3380-43c0-bd7d-2fded82ff47f-kube-api-access-m5kgg\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:16.168200 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.168142    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6rzw\" (UniqueName: \"kubernetes.io/projected/cdd66c31-3562-4381-bdbd-aaaf2a76d679-kube-api-access-d6rzw\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:16.168200 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.168152    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:16.168200 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.168160    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8137ccb6-4348-4057-b7af-66bb5ed3d02f-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:16.168200 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.168169    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v67fw\" (UniqueName: \"kubernetes.io/projected/8137ccb6-4348-4057-b7af-66bb5ed3d02f-kube-api-access-v67fw\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:16.637152 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637112    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"]
Sep 07 09:14:16.637787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637766    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d535cf66-3380-43c0-bd7d-2fded82ff47f" containerName="payload-processing"
Sep 07 09:14:16.637843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637793    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d535cf66-3380-43c0-bd7d-2fded82ff47f" containerName="payload-processing"
Sep 07 09:14:16.637843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637814    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd66c31-3562-4381-bdbd-aaaf2a76d679" containerName="payload-pre-processing"
Sep 07 09:14:16.637843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637823    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd66c31-3562-4381-bdbd-aaaf2a76d679" containerName="payload-pre-processing"
Sep 07 09:14:16.637941 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637866    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8137ccb6-4348-4057-b7af-66bb5ed3d02f" containerName="maas-api"
Sep 07 09:14:16.637941 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637875    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8137ccb6-4348-4057-b7af-66bb5ed3d02f" containerName="maas-api"
Sep 07 09:14:16.638001 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637955    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8137ccb6-4348-4057-b7af-66bb5ed3d02f" containerName="maas-api"
Sep 07 09:14:16.638001 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637967    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d535cf66-3380-43c0-bd7d-2fded82ff47f" containerName="payload-processing"
Sep 07 09:14:16.638001 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.637983    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd66c31-3562-4381-bdbd-aaaf2a76d679" containerName="payload-pre-processing"
Sep 07 09:14:16.642823 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.642801    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.645900 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.645873    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dc11d6be-openshift-default-dockercfg-vm6lq\""
Sep 07 09:14:16.652393 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.652371    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"]
Sep 07 09:14:16.774818 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.774774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-credential-socket\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.775010 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.774830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-certs\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.775010 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.774928    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr2jq\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-kube-api-access-jr2jq\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.775010 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.774963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-socket\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.775117 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.775036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-data\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.775117 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.775066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-envoy\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.775117 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.775091    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-token\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.775212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.775115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-podinfo\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.775212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.775180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istiod-ca-cert\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.821055 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.821024    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"
Sep 07 09:14:16.821055 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.821043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s" event={"ID":"d535cf66-3380-43c0-bd7d-2fded82ff47f","Type":"ContainerDied","Data":"f2c8333a7ecf0fe186d4f4c729e465cd50ffe5b93cda759bb2493efd35a81c9d"}
Sep 07 09:14:16.821340 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.821112    2563 scope.go:117] "RemoveContainer" containerID="e4c51c0dd05e1f188bda6120941ed8409d6ebb55dd53cf20247ca07a73c9d1e0"
Sep 07 09:14:16.822505 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.822489    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"
Sep 07 09:14:16.822591 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.822506    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7" event={"ID":"cdd66c31-3562-4381-bdbd-aaaf2a76d679","Type":"ContainerDied","Data":"d292ba38fd1fa645f0dad0642e5206d592333bf37fe2f879965cd0b37ec86f11"}
Sep 07 09:14:16.824349 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.824329    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"
Sep 07 09:14:16.824465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.824323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2" event={"ID":"8137ccb6-4348-4057-b7af-66bb5ed3d02f","Type":"ContainerDied","Data":"e6a9ebe0970c32945072700dd0c610647b6807dbf19218fc98bee104ba9bb6bd"}
Sep 07 09:14:16.831412 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.831396    2563 scope.go:117] "RemoveContainer" containerID="0c3d13c7333247f87d42d60ce0eaa933a204b9ee7ed2534c9ab390cfe340dd4e"
Sep 07 09:14:16.839432 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.839408    2563 scope.go:117] "RemoveContainer" containerID="79d72e868e60a80d29a07db4ce996c2f3ae0bca55b8f08ef333c959d4a1f7d7d"
Sep 07 09:14:16.845721 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.845690    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"]
Sep 07 09:14:16.849226 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.849206    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7928b616-686546d77d-8gpn7"]
Sep 07 09:14:16.859344 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.859317    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"]
Sep 07 09:14:16.863367 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.863342    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7928b616-594b5f4fc4-mlf6s"]
Sep 07 09:14:16.873053 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.873028    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"]
Sep 07 09:14:16.875667 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.875645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr2jq\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-kube-api-access-jr2jq\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.875765 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.875677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-socket\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.875765 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.875736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-data\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.875851 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.875766    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-envoy\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.875894 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.875854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-token\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.875942 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.875920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-podinfo\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.875996 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.875961    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istiod-ca-cert\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.876049 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.875993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-credential-socket\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.876049 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.876033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-certs\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.877536 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.877513    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7928b616-b4b768bb8-7g2h2"]
Sep 07 09:14:16.878999 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.878973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-data\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.879107 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.879083    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-envoy\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.879179 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.879125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-socket\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.879179 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.879145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-credential-socket\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.879179 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.879168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-certs\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.879376 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.879353    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istiod-ca-cert\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.879560 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.879541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-podinfo\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.883611 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.883579    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-token\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.883719 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.883703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr2jq\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-kube-api-access-jr2jq\") pod \"e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") " pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:16.954382 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:16.954346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:17.229185 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:17.229158    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"]
Sep 07 09:14:17.231399 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:14:17.231365    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod100dd3bb_139c_4bf3_9f3c_33e511cf9914.slice/crio-69a41ed11d05b7e16780a501fec6b4fb892b683123b2521b9c294bd131085835 WatchSource:0}: Error finding container 69a41ed11d05b7e16780a501fec6b4fb892b683123b2521b9c294bd131085835: Status 404 returned error can't find the container with id 69a41ed11d05b7e16780a501fec6b4fb892b683123b2521b9c294bd131085835
Sep 07 09:14:17.233709 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:17.233666    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:17.233805 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:17.233741    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:17.233805 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:17.233771    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:17.834696 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:17.834658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" event={"ID":"100dd3bb-139c-4bf3-9f3c-33e511cf9914","Type":"ContainerStarted","Data":"36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec"}
Sep 07 09:14:17.835166 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:17.834723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" event={"ID":"100dd3bb-139c-4bf3-9f3c-33e511cf9914","Type":"ContainerStarted","Data":"69a41ed11d05b7e16780a501fec6b4fb892b683123b2521b9c294bd131085835"}
Sep 07 09:14:17.856910 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:17.856858    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" podStartSLOduration=1.856841824 podStartE2EDuration="1.856841824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:17.854932791 +0000 UTC m=+915.824103397" watchObservedRunningTime="2026-09-07 09:14:17.856841824 +0000 UTC m=+915.826012429"
Sep 07 09:14:17.955468 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:17.955434    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:18.465646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:18.465602    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8137ccb6-4348-4057-b7af-66bb5ed3d02f" path="/var/lib/kubelet/pods/8137ccb6-4348-4057-b7af-66bb5ed3d02f/volumes"
Sep 07 09:14:18.466090 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:18.466067    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd66c31-3562-4381-bdbd-aaaf2a76d679" path="/var/lib/kubelet/pods/cdd66c31-3562-4381-bdbd-aaaf2a76d679/volumes"
Sep 07 09:14:18.466488 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:18.466467    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d535cf66-3380-43c0-bd7d-2fded82ff47f" path="/var/lib/kubelet/pods/d535cf66-3380-43c0-bd7d-2fded82ff47f/volumes"
Sep 07 09:14:18.960155 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:18.960122    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:19.844420 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:19.844382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:19.845471 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:19.845450    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:20.918049 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:20.918013    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"]
Sep 07 09:14:20.923104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:20.923078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:20.926072 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:20.926047    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-dc11d6be-dockercfg-w6gvb\""
Sep 07 09:14:20.926210 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:20.926081    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-dc11d6be\""
Sep 07 09:14:20.933491 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:20.933466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"]
Sep 07 09:14:20.951362 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:20.951330    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"]
Sep 07 09:14:20.954972 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:20.954939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:20.969478 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:20.969450    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"]
Sep 07 09:14:21.021271 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.021234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e14553ca-81bb-46bb-82db-5ef1c86918a2-plugins-config-volume\") pod \"payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh\" (UID: \"e14553ca-81bb-46bb-82db-5ef1c86918a2\") " pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:21.021471 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.021341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqvl7\" (UniqueName: \"kubernetes.io/projected/e14553ca-81bb-46bb-82db-5ef1c86918a2-kube-api-access-vqvl7\") pod \"payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh\" (UID: \"e14553ca-81bb-46bb-82db-5ef1c86918a2\") " pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:21.021471 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.021392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdc4g\" (UniqueName: \"kubernetes.io/projected/0f2062b1-b052-46c2-849a-74e2b70b1398-kube-api-access-gdc4g\") pod \"payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk\" (UID: \"0f2062b1-b052-46c2-849a-74e2b70b1398\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:21.021471 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.021445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2062b1-b052-46c2-849a-74e2b70b1398-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk\" (UID: \"0f2062b1-b052-46c2-849a-74e2b70b1398\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:21.123051 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.123001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2062b1-b052-46c2-849a-74e2b70b1398-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk\" (UID: \"0f2062b1-b052-46c2-849a-74e2b70b1398\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:21.123202 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.123145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e14553ca-81bb-46bb-82db-5ef1c86918a2-plugins-config-volume\") pod \"payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh\" (UID: \"e14553ca-81bb-46bb-82db-5ef1c86918a2\") " pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:21.123246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.123214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqvl7\" (UniqueName: \"kubernetes.io/projected/e14553ca-81bb-46bb-82db-5ef1c86918a2-kube-api-access-vqvl7\") pod \"payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh\" (UID: \"e14553ca-81bb-46bb-82db-5ef1c86918a2\") " pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:21.123301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.123261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdc4g\" (UniqueName: \"kubernetes.io/projected/0f2062b1-b052-46c2-849a-74e2b70b1398-kube-api-access-gdc4g\") pod \"payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk\" (UID: \"0f2062b1-b052-46c2-849a-74e2b70b1398\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:21.123943 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.123855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2062b1-b052-46c2-849a-74e2b70b1398-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk\" (UID: \"0f2062b1-b052-46c2-849a-74e2b70b1398\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:21.124071 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.123966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e14553ca-81bb-46bb-82db-5ef1c86918a2-plugins-config-volume\") pod \"payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh\" (UID: \"e14553ca-81bb-46bb-82db-5ef1c86918a2\") " pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:21.131257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.131233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqvl7\" (UniqueName: \"kubernetes.io/projected/e14553ca-81bb-46bb-82db-5ef1c86918a2-kube-api-access-vqvl7\") pod \"payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh\" (UID: \"e14553ca-81bb-46bb-82db-5ef1c86918a2\") " pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:21.131661 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.131639    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdc4g\" (UniqueName: \"kubernetes.io/projected/0f2062b1-b052-46c2-849a-74e2b70b1398-kube-api-access-gdc4g\") pod \"payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk\" (UID: \"0f2062b1-b052-46c2-849a-74e2b70b1398\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:21.185631 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.185602    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:14:21.236657 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.236622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:21.309798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.309753    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:21.326174 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-certs\") pod \"19189c89-4e3f-4153-8ad8-5c29aee3848f\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") "
Sep 07 09:14:21.326174 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326174    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-credential-socket\") pod \"19189c89-4e3f-4153-8ad8-5c29aee3848f\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") "
Sep 07 09:14:21.326446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326215    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-socket\") pod \"19189c89-4e3f-4153-8ad8-5c29aee3848f\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") "
Sep 07 09:14:21.326446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326297    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-envoy\") pod \"19189c89-4e3f-4153-8ad8-5c29aee3848f\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") "
Sep 07 09:14:21.326446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326340    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzknl\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-kube-api-access-nzknl\") pod \"19189c89-4e3f-4153-8ad8-5c29aee3848f\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") "
Sep 07 09:14:21.326446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326384    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-token\") pod \"19189c89-4e3f-4153-8ad8-5c29aee3848f\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") "
Sep 07 09:14:21.326446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326415    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-podinfo\") pod \"19189c89-4e3f-4153-8ad8-5c29aee3848f\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") "
Sep 07 09:14:21.326658 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326458    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19189c89-4e3f-4153-8ad8-5c29aee3848f" (UID: "19189c89-4e3f-4153-8ad8-5c29aee3848f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:21.326658 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326645    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19189c89-4e3f-4153-8ad8-5c29aee3848f" (UID: "19189c89-4e3f-4153-8ad8-5c29aee3848f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:21.326761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326693    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19189c89-4e3f-4153-8ad8-5c29aee3848f-istiod-ca-cert\") pod \"19189c89-4e3f-4153-8ad8-5c29aee3848f\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") "
Sep 07 09:14:21.326816 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.326778    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-data\") pod \"19189c89-4e3f-4153-8ad8-5c29aee3848f\" (UID: \"19189c89-4e3f-4153-8ad8-5c29aee3848f\") "
Sep 07 09:14:21.327124 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.327073    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19189c89-4e3f-4153-8ad8-5c29aee3848f" (UID: "19189c89-4e3f-4153-8ad8-5c29aee3848f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:21.327124 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.327104    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19189c89-4e3f-4153-8ad8-5c29aee3848f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19189c89-4e3f-4153-8ad8-5c29aee3848f" (UID: "19189c89-4e3f-4153-8ad8-5c29aee3848f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:14:21.327263 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.327111    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:21.327263 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.327155    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:21.327391 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.327351    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-data" (OuterVolumeSpecName: "istio-data") pod "19189c89-4e3f-4153-8ad8-5c29aee3848f" (UID: "19189c89-4e3f-4153-8ad8-5c29aee3848f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:21.329306 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.329245    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19189c89-4e3f-4153-8ad8-5c29aee3848f" (UID: "19189c89-4e3f-4153-8ad8-5c29aee3848f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:14:21.329306 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.329254    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-token" (OuterVolumeSpecName: "istio-token") pod "19189c89-4e3f-4153-8ad8-5c29aee3848f" (UID: "19189c89-4e3f-4153-8ad8-5c29aee3848f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:21.329306 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.329267    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-kube-api-access-nzknl" (OuterVolumeSpecName: "kube-api-access-nzknl") pod "19189c89-4e3f-4153-8ad8-5c29aee3848f" (UID: "19189c89-4e3f-4153-8ad8-5c29aee3848f"). InnerVolumeSpecName "kube-api-access-nzknl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:21.329528 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.329507    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19189c89-4e3f-4153-8ad8-5c29aee3848f" (UID: "19189c89-4e3f-4153-8ad8-5c29aee3848f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:21.428018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.427932    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:21.428018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.427963    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:21.428018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.427973    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:21.428018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.427982    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzknl\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-kube-api-access-nzknl\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:21.428018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.427992    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:21.428018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.428002    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19189c89-4e3f-4153-8ad8-5c29aee3848f-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:21.428018 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.428015    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19189c89-4e3f-4153-8ad8-5c29aee3848f-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:21.506131 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.506104    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"]
Sep 07 09:14:21.507810 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:14:21.507781    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f2062b1_b052_46c2_849a_74e2b70b1398.slice/crio-292001e6b2ba157aa85d44d53405dedeff11545a664fc9d8de58adcc576e70eb WatchSource:0}: Error finding container 292001e6b2ba157aa85d44d53405dedeff11545a664fc9d8de58adcc576e70eb: Status 404 returned error can't find the container with id 292001e6b2ba157aa85d44d53405dedeff11545a664fc9d8de58adcc576e70eb
Sep 07 09:14:21.590970 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.590941    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"]
Sep 07 09:14:21.593191 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:14:21.593157    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode14553ca_81bb_46bb_82db_5ef1c86918a2.slice/crio-e0b2ea2278a01e408d5fb6b531115071fdb18aa6ea27e60ca2ee9dd2cc77cfb3 WatchSource:0}: Error finding container e0b2ea2278a01e408d5fb6b531115071fdb18aa6ea27e60ca2ee9dd2cc77cfb3: Status 404 returned error can't find the container with id e0b2ea2278a01e408d5fb6b531115071fdb18aa6ea27e60ca2ee9dd2cc77cfb3
Sep 07 09:14:21.857612 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.857560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk" event={"ID":"0f2062b1-b052-46c2-849a-74e2b70b1398","Type":"ContainerStarted","Data":"da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c"}
Sep 07 09:14:21.857815 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.857629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk" event={"ID":"0f2062b1-b052-46c2-849a-74e2b70b1398","Type":"ContainerStarted","Data":"292001e6b2ba157aa85d44d53405dedeff11545a664fc9d8de58adcc576e70eb"}
Sep 07 09:14:21.857815 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.857658    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:21.859646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.859614    2563 generic.go:356] "Generic (PLEG): container finished" podID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerID="3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2" exitCode=0
Sep 07 09:14:21.859798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.859664    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"
Sep 07 09:14:21.859798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.859697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" event={"ID":"19189c89-4e3f-4153-8ad8-5c29aee3848f","Type":"ContainerDied","Data":"3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2"}
Sep 07 09:14:21.859798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.859758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm" event={"ID":"19189c89-4e3f-4153-8ad8-5c29aee3848f","Type":"ContainerDied","Data":"c38df0b2a0e0e02bd96ca5c88b7ea8e344e7b6557df4c432545104e6d0937aaa"}
Sep 07 09:14:21.859798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.859789    2563 scope.go:117] "RemoveContainer" containerID="3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2"
Sep 07 09:14:21.861713 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.861684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh" event={"ID":"e14553ca-81bb-46bb-82db-5ef1c86918a2","Type":"ContainerStarted","Data":"c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa"}
Sep 07 09:14:21.861856 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.861738    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh" event={"ID":"e14553ca-81bb-46bb-82db-5ef1c86918a2","Type":"ContainerStarted","Data":"e0b2ea2278a01e408d5fb6b531115071fdb18aa6ea27e60ca2ee9dd2cc77cfb3"}
Sep 07 09:14:21.861931 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.861896    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:21.869971 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.869947    2563 scope.go:117] "RemoveContainer" containerID="18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107"
Sep 07 09:14:21.876570 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.876525    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk" podStartSLOduration=1.876512703 podStartE2EDuration="1.876512703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:21.874529903 +0000 UTC m=+919.843700509" watchObservedRunningTime="2026-09-07 09:14:21.876512703 +0000 UTC m=+919.845683307"
Sep 07 09:14:21.880221 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.880203    2563 scope.go:117] "RemoveContainer" containerID="3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2"
Sep 07 09:14:21.880517 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:14:21.880497    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2\": container with ID starting with 3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2 not found: ID does not exist" containerID="3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2"
Sep 07 09:14:21.880589 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.880524    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2"} err="failed to get container status \"3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2\": rpc error: code = NotFound desc = could not find container \"3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2\": container with ID starting with 3111d5d3731be201d2a94d843598ba03c489f70adda7e6f5925715558a4a9fb2 not found: ID does not exist"
Sep 07 09:14:21.880589 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.880541    2563 scope.go:117] "RemoveContainer" containerID="18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107"
Sep 07 09:14:21.880819 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:14:21.880794    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107\": container with ID starting with 18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107 not found: ID does not exist" containerID="18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107"
Sep 07 09:14:21.880879 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.880826    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107"} err="failed to get container status \"18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107\": rpc error: code = NotFound desc = could not find container \"18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107\": container with ID starting with 18e061fcafc3f0f31663922bb14cd030091399dc330aa96a00b3a05e6ddd9107 not found: ID does not exist"
Sep 07 09:14:21.895544 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.895504    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh" podStartSLOduration=1.8954895760000001 podStartE2EDuration="1.895489576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:21.893093649 +0000 UTC m=+919.862264256" watchObservedRunningTime="2026-09-07 09:14:21.895489576 +0000 UTC m=+919.864660180"
Sep 07 09:14:21.909016 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.908993    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"]
Sep 07 09:14:21.915998 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:21.915971    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-7928b616-gw-openshift-default-674d8576c6-z6rbm"]
Sep 07 09:14:22.466952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:22.466907    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" path="/var/lib/kubelet/pods/19189c89-4e3f-4153-8ad8-5c29aee3848f/volumes"
Sep 07 09:14:32.871118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:32.871081    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:32.871584 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:32.871259    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:38.956061 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:38.956026    2563 generic.go:356] "Generic (PLEG): container finished" podID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerID="36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec" exitCode=0
Sep 07 09:14:38.956495 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:38.956096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" event={"ID":"100dd3bb-139c-4bf3-9f3c-33e511cf9914","Type":"ContainerDied","Data":"36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec"}
Sep 07 09:14:38.956540 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:38.956516    2563 scope.go:117] "RemoveContainer" containerID="36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec"
Sep 07 09:14:38.957581 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:38.957553    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:38.957675 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:38.957657    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:38.957730 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:38.957709    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:39.967218 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:39.967161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" event={"ID":"100dd3bb-139c-4bf3-9f3c-33e511cf9914","Type":"ContainerStarted","Data":"778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296"}
Sep 07 09:14:40.955077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:40.955032    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:40.955556 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:40.955526    2563 patch_prober.go:28] interesting pod/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 07 09:14:40.955676 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:40.955595    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 07 09:14:41.954993 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:41.954951    2563 patch_prober.go:28] interesting pod/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 07 09:14:41.955626 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:41.955587    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 07 09:14:42.955405 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:42.955357    2563 patch_prober.go:28] interesting pod/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 07 09:14:42.955939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:42.955445    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 07 09:14:43.955338 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:43.955274    2563 patch_prober.go:28] interesting pod/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 07 09:14:43.955555 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:43.955367    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 07 09:14:44.955493 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:44.955455    2563 patch_prober.go:28] interesting pod/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 07 09:14:44.955683 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:44.955525    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 07 09:14:45.959028 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:45.958993    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:45.959430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:45.959388    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:45.960100 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:45.960079    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:50.928013 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:50.927974    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"]
Sep 07 09:14:50.928611 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:50.928203    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh" podUID="e14553ca-81bb-46bb-82db-5ef1c86918a2" containerName="payload-processing" containerID="cri-o://c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa" gracePeriod=30
Sep 07 09:14:50.939249 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:50.939218    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"]
Sep 07 09:14:50.939559 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:50.939512    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk" podUID="0f2062b1-b052-46c2-849a-74e2b70b1398" containerName="payload-pre-processing" containerID="cri-o://da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c" gracePeriod=30
Sep 07 09:14:51.223689 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.223659    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:51.226963 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.226935    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:51.329815 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.329779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e14553ca-81bb-46bb-82db-5ef1c86918a2-plugins-config-volume\") pod \"e14553ca-81bb-46bb-82db-5ef1c86918a2\" (UID: \"e14553ca-81bb-46bb-82db-5ef1c86918a2\") "
Sep 07 09:14:51.329996 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.329865    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2062b1-b052-46c2-849a-74e2b70b1398-plugins-config-volume\") pod \"0f2062b1-b052-46c2-849a-74e2b70b1398\" (UID: \"0f2062b1-b052-46c2-849a-74e2b70b1398\") "
Sep 07 09:14:51.329996 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.329988    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqvl7\" (UniqueName: \"kubernetes.io/projected/e14553ca-81bb-46bb-82db-5ef1c86918a2-kube-api-access-vqvl7\") pod \"e14553ca-81bb-46bb-82db-5ef1c86918a2\" (UID: \"e14553ca-81bb-46bb-82db-5ef1c86918a2\") "
Sep 07 09:14:51.330087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.330015    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdc4g\" (UniqueName: \"kubernetes.io/projected/0f2062b1-b052-46c2-849a-74e2b70b1398-kube-api-access-gdc4g\") pod \"0f2062b1-b052-46c2-849a-74e2b70b1398\" (UID: \"0f2062b1-b052-46c2-849a-74e2b70b1398\") "
Sep 07 09:14:51.330254 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.330231    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e14553ca-81bb-46bb-82db-5ef1c86918a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e14553ca-81bb-46bb-82db-5ef1c86918a2" (UID: "e14553ca-81bb-46bb-82db-5ef1c86918a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:14:51.330315 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.330259    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f2062b1-b052-46c2-849a-74e2b70b1398-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f2062b1-b052-46c2-849a-74e2b70b1398" (UID: "0f2062b1-b052-46c2-849a-74e2b70b1398"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:14:51.330440 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.330418    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e14553ca-81bb-46bb-82db-5ef1c86918a2-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:51.330440 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.330438    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2062b1-b052-46c2-849a-74e2b70b1398-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:51.332107 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.332087    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f2062b1-b052-46c2-849a-74e2b70b1398-kube-api-access-gdc4g" (OuterVolumeSpecName: "kube-api-access-gdc4g") pod "0f2062b1-b052-46c2-849a-74e2b70b1398" (UID: "0f2062b1-b052-46c2-849a-74e2b70b1398"). InnerVolumeSpecName "kube-api-access-gdc4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:51.332185 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.332117    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e14553ca-81bb-46bb-82db-5ef1c86918a2-kube-api-access-vqvl7" (OuterVolumeSpecName: "kube-api-access-vqvl7") pod "e14553ca-81bb-46bb-82db-5ef1c86918a2" (UID: "e14553ca-81bb-46bb-82db-5ef1c86918a2"). InnerVolumeSpecName "kube-api-access-vqvl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:51.431930 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.431891    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqvl7\" (UniqueName: \"kubernetes.io/projected/e14553ca-81bb-46bb-82db-5ef1c86918a2-kube-api-access-vqvl7\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:51.431930 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:51.431923    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdc4g\" (UniqueName: \"kubernetes.io/projected/0f2062b1-b052-46c2-849a-74e2b70b1398-kube-api-access-gdc4g\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:52.032925 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.032890    2563 generic.go:356] "Generic (PLEG): container finished" podID="0f2062b1-b052-46c2-849a-74e2b70b1398" containerID="da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c" exitCode=0
Sep 07 09:14:52.033426 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.032939    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"
Sep 07 09:14:52.033426 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.032979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk" event={"ID":"0f2062b1-b052-46c2-849a-74e2b70b1398","Type":"ContainerDied","Data":"da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c"}
Sep 07 09:14:52.033426 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.033048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk" event={"ID":"0f2062b1-b052-46c2-849a-74e2b70b1398","Type":"ContainerDied","Data":"292001e6b2ba157aa85d44d53405dedeff11545a664fc9d8de58adcc576e70eb"}
Sep 07 09:14:52.033426 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.033084    2563 scope.go:117] "RemoveContainer" containerID="da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c"
Sep 07 09:14:52.034565 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.034371    2563 generic.go:356] "Generic (PLEG): container finished" podID="e14553ca-81bb-46bb-82db-5ef1c86918a2" containerID="c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa" exitCode=0
Sep 07 09:14:52.034565 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.034415    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"
Sep 07 09:14:52.034565 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.034411    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh" event={"ID":"e14553ca-81bb-46bb-82db-5ef1c86918a2","Type":"ContainerDied","Data":"c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa"}
Sep 07 09:14:52.034565 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.034526    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh" event={"ID":"e14553ca-81bb-46bb-82db-5ef1c86918a2","Type":"ContainerDied","Data":"e0b2ea2278a01e408d5fb6b531115071fdb18aa6ea27e60ca2ee9dd2cc77cfb3"}
Sep 07 09:14:52.043632 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.043370    2563 scope.go:117] "RemoveContainer" containerID="da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c"
Sep 07 09:14:52.043775 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:14:52.043750    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c\": container with ID starting with da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c not found: ID does not exist" containerID="da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c"
Sep 07 09:14:52.043841 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.043787    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c"} err="failed to get container status \"da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c\": rpc error: code = NotFound desc = could not find container \"da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c\": container with ID starting with da5e9749403a2cc8afd6bffefb9c385bdb62cd68901ac157b7f17a56d09bc41c not found: ID does not exist"
Sep 07 09:14:52.043841 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.043811    2563 scope.go:117] "RemoveContainer" containerID="c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa"
Sep 07 09:14:52.051992 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.051976    2563 scope.go:117] "RemoveContainer" containerID="c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa"
Sep 07 09:14:52.052236 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:14:52.052217    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa\": container with ID starting with c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa not found: ID does not exist" containerID="c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa"
Sep 07 09:14:52.052298 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.052243    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa"} err="failed to get container status \"c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa\": rpc error: code = NotFound desc = could not find container \"c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa\": container with ID starting with c86e0ffb60b586dbd15627e7c9183305010a5e99f66f2c514a6f6a65acf4ecaa not found: ID does not exist"
Sep 07 09:14:52.058534 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.058514    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"]
Sep 07 09:14:52.066091 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.066065    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dc11d6be-cbd97bdfd-slpvk"]
Sep 07 09:14:52.076519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.076496    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"]
Sep 07 09:14:52.079867 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.079844    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dc11d6be-64b9d5f6b7-8q6nh"]
Sep 07 09:14:52.186025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.185992    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"]
Sep 07 09:14:52.186251 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.186225    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy" containerID="cri-o://778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296" gracePeriod=30
Sep 07 09:14:52.465297 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.465253    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f2062b1-b052-46c2-849a-74e2b70b1398" path="/var/lib/kubelet/pods/0f2062b1-b052-46c2-849a-74e2b70b1398/volumes"
Sep 07 09:14:52.465655 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.465641    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e14553ca-81bb-46bb-82db-5ef1c86918a2" path="/var/lib/kubelet/pods/e14553ca-81bb-46bb-82db-5ef1c86918a2/volumes"
Sep 07 09:14:52.877681 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.877648    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"]
Sep 07 09:14:52.878044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878031    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy"
Sep 07 09:14:52.878092 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878046    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy"
Sep 07 09:14:52.878092 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878063    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy"
Sep 07 09:14:52.878092 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878069    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy"
Sep 07 09:14:52.878092 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878077    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14553ca-81bb-46bb-82db-5ef1c86918a2" containerName="payload-processing"
Sep 07 09:14:52.878092 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878083    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14553ca-81bb-46bb-82db-5ef1c86918a2" containerName="payload-processing"
Sep 07 09:14:52.878255 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878094    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f2062b1-b052-46c2-849a-74e2b70b1398" containerName="payload-pre-processing"
Sep 07 09:14:52.878255 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878100    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f2062b1-b052-46c2-849a-74e2b70b1398" containerName="payload-pre-processing"
Sep 07 09:14:52.878255 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878156    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy"
Sep 07 09:14:52.878255 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878166    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f2062b1-b052-46c2-849a-74e2b70b1398" containerName="payload-pre-processing"
Sep 07 09:14:52.878255 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.878176    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14553ca-81bb-46bb-82db-5ef1c86918a2" containerName="payload-processing"
Sep 07 09:14:52.881312 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.881277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:52.883684 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.883662    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-863fd8cf-openshift-default-dockercfg-vbj6h\""
Sep 07 09:14:52.891763 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.891740    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"]
Sep 07 09:14:52.944639 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.944597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-data\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:52.944835 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.944651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-certs\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:52.944835 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.944711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-socket\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:52.944969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.944826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-token\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:52.944969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.944862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xx9h\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-kube-api-access-7xx9h\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:52.944969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.944904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istiod-ca-cert\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:52.944969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.944934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-credential-socket\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:52.945121 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.944980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-envoy\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:52.945121 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:52.945036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-podinfo\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.045826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.045790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-certs\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046333 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.045835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-socket\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046333 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.045894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-token\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046333 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.045918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xx9h\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-kube-api-access-7xx9h\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046333 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.045951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istiod-ca-cert\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046333 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.045981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-credential-socket\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046333 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.046018    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-envoy\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046333 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.046089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-podinfo\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046333 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.046162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-data\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046333 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.046251    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-certs\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046857 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.046404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-credential-socket\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046857 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.046613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-socket\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046857 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.046636    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-data\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.046857 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.046704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istiod-ca-cert\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.048429 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.048408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-envoy\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.048763 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.048734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-podinfo\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.057876 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.057855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-token\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.057982 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.057962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xx9h\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-kube-api-access-7xx9h\") pod \"e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") " pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.192168 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.192076    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:53.468818 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.468786    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"]
Sep 07 09:14:53.470541 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:14:53.470515    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb00b28d_deb7_4809_a3e7_d81a108ad57e.slice/crio-f1ebb7e911743858f75109967e7cc3d1b0cf08f93a6e6d99a38ab1721010b55a WatchSource:0}: Error finding container f1ebb7e911743858f75109967e7cc3d1b0cf08f93a6e6d99a38ab1721010b55a: Status 404 returned error can't find the container with id f1ebb7e911743858f75109967e7cc3d1b0cf08f93a6e6d99a38ab1721010b55a
Sep 07 09:14:53.472653 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.472618    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:53.472718 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.472690    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:53.472761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:53.472718    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:14:54.049667 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:54.049611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" event={"ID":"bb00b28d-deb7-4809-a3e7-d81a108ad57e","Type":"ContainerStarted","Data":"1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca"}
Sep 07 09:14:54.050130 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:54.049681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" event={"ID":"bb00b28d-deb7-4809-a3e7-d81a108ad57e","Type":"ContainerStarted","Data":"f1ebb7e911743858f75109967e7cc3d1b0cf08f93a6e6d99a38ab1721010b55a"}
Sep 07 09:14:54.068515 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:54.068277    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" podStartSLOduration=2.068254357 podStartE2EDuration="2.068254357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:54.068111372 +0000 UTC m=+952.037281989" watchObservedRunningTime="2026-09-07 09:14:54.068254357 +0000 UTC m=+952.037424966"
Sep 07 09:14:54.192463 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:54.192430    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:54.197565 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:54.197535    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:55.055125 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:55.055093    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:55.056377 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:55.056348    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:14:56.502979 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.502940    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"]
Sep 07 09:14:56.503674 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.503653    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="19189c89-4e3f-4153-8ad8-5c29aee3848f" containerName="istio-proxy"
Sep 07 09:14:56.506566 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.506545    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.509465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.509446    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-863fd8cf-serving-cert\""
Sep 07 09:14:56.519400 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.519375    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"]
Sep 07 09:14:56.584891 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.584850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39e87ab-91ed-4200-9fba-855aa30318b2-openshift-service-ca\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.584891 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.584889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr7zf\" (UniqueName: \"kubernetes.io/projected/f39e87ab-91ed-4200-9fba-855aa30318b2-kube-api-access-dr7zf\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.585115 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.585001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-metrics-certs\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.585115 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.585086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-maas-api-tls\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.686108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.686068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39e87ab-91ed-4200-9fba-855aa30318b2-openshift-service-ca\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.686108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.686108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr7zf\" (UniqueName: \"kubernetes.io/projected/f39e87ab-91ed-4200-9fba-855aa30318b2-kube-api-access-dr7zf\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.686360 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.686181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-metrics-certs\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.686360 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.686264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-maas-api-tls\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.686453 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:14:56.686405    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-serving-cert: secret "maas-api-e2e-derive-863fd8cf-serving-cert" not found
Sep 07 09:14:56.686494 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:14:56.686488    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-maas-api-tls podName:f39e87ab-91ed-4200-9fba-855aa30318b2 nodeName:}" failed. No retries permitted until 2026-09-07 09:14:57.186466746 +0000 UTC m=+955.155637329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-maas-api-tls") pod "maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42" (UID: "f39e87ab-91ed-4200-9fba-855aa30318b2") : secret "maas-api-e2e-derive-863fd8cf-serving-cert" not found
Sep 07 09:14:56.686897 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.686875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39e87ab-91ed-4200-9fba-855aa30318b2-openshift-service-ca\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.688606 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.688584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-metrics-certs\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:56.694576 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:56.694550    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr7zf\" (UniqueName: \"kubernetes.io/projected/f39e87ab-91ed-4200-9fba-855aa30318b2-kube-api-access-dr7zf\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:57.192350 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.192309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-maas-api-tls\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:57.194878 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.194843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-maas-api-tls\") pod \"maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:57.220010 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.219982    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"]
Sep 07 09:14:57.222769 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.222750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:14:57.225914 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.225893    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-863fd8cf-dockercfg-46w7t\""
Sep 07 09:14:57.226356 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.226340    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-863fd8cf\""
Sep 07 09:14:57.238429 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.238403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"]
Sep 07 09:14:57.293947 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.293913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60920cf9-4df6-40e9-98dd-d54e51b848e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr\" (UID: \"60920cf9-4df6-40e9-98dd-d54e51b848e4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:14:57.294112 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.293979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29sz8\" (UniqueName: \"kubernetes.io/projected/60920cf9-4df6-40e9-98dd-d54e51b848e4-kube-api-access-29sz8\") pod \"payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr\" (UID: \"60920cf9-4df6-40e9-98dd-d54e51b848e4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:14:57.395338 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.395303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29sz8\" (UniqueName: \"kubernetes.io/projected/60920cf9-4df6-40e9-98dd-d54e51b848e4-kube-api-access-29sz8\") pod \"payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr\" (UID: \"60920cf9-4df6-40e9-98dd-d54e51b848e4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:14:57.395542 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.395423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60920cf9-4df6-40e9-98dd-d54e51b848e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr\" (UID: \"60920cf9-4df6-40e9-98dd-d54e51b848e4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:14:57.396124 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.396102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60920cf9-4df6-40e9-98dd-d54e51b848e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr\" (UID: \"60920cf9-4df6-40e9-98dd-d54e51b848e4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:14:57.403825 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.403799    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29sz8\" (UniqueName: \"kubernetes.io/projected/60920cf9-4df6-40e9-98dd-d54e51b848e4-kube-api-access-29sz8\") pod \"payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr\" (UID: \"60920cf9-4df6-40e9-98dd-d54e51b848e4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:14:57.420037 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.420016    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:57.422989 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.422973    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:57.495814 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.495785    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-data\") pod \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") "
Sep 07 09:14:57.495969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.495850    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-token\") pod \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") "
Sep 07 09:14:57.495969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.495897    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr2jq\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-kube-api-access-jr2jq\") pod \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") "
Sep 07 09:14:57.495969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.495916    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-podinfo\") pod \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") "
Sep 07 09:14:57.496077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.495970    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-socket\") pod \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") "
Sep 07 09:14:57.496077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.496018    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-envoy\") pod \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") "
Sep 07 09:14:57.496077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.496034    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istiod-ca-cert\") pod \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") "
Sep 07 09:14:57.496077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.496056    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-credential-socket\") pod \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") "
Sep 07 09:14:57.496077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.496071    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-certs\") pod \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\" (UID: \"100dd3bb-139c-4bf3-9f3c-33e511cf9914\") "
Sep 07 09:14:57.496932 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.496551    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-data" (OuterVolumeSpecName: "istio-data") pod "100dd3bb-139c-4bf3-9f3c-33e511cf9914" (UID: "100dd3bb-139c-4bf3-9f3c-33e511cf9914"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:57.496932 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.496574    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "100dd3bb-139c-4bf3-9f3c-33e511cf9914" (UID: "100dd3bb-139c-4bf3-9f3c-33e511cf9914"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:57.496932 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.496794    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "100dd3bb-139c-4bf3-9f3c-33e511cf9914" (UID: "100dd3bb-139c-4bf3-9f3c-33e511cf9914"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:57.497167 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.497124    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "100dd3bb-139c-4bf3-9f3c-33e511cf9914" (UID: "100dd3bb-139c-4bf3-9f3c-33e511cf9914"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:57.497843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.497660    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "100dd3bb-139c-4bf3-9f3c-33e511cf9914" (UID: "100dd3bb-139c-4bf3-9f3c-33e511cf9914"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:14:57.499663 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.499624    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-token" (OuterVolumeSpecName: "istio-token") pod "100dd3bb-139c-4bf3-9f3c-33e511cf9914" (UID: "100dd3bb-139c-4bf3-9f3c-33e511cf9914"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:57.499773 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.499661    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-kube-api-access-jr2jq" (OuterVolumeSpecName: "kube-api-access-jr2jq") pod "100dd3bb-139c-4bf3-9f3c-33e511cf9914" (UID: "100dd3bb-139c-4bf3-9f3c-33e511cf9914"). InnerVolumeSpecName "kube-api-access-jr2jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:57.500089 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.500067    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "100dd3bb-139c-4bf3-9f3c-33e511cf9914" (UID: "100dd3bb-139c-4bf3-9f3c-33e511cf9914"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:14:57.500461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.500439    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "100dd3bb-139c-4bf3-9f3c-33e511cf9914" (UID: "100dd3bb-139c-4bf3-9f3c-33e511cf9914"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:14:57.575427 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.575385    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:14:57.597316 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.597273    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:57.597316 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.597315    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:57.597519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.597325    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:57.597519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.597335    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:57.597519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.597346    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:57.597519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.597355    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:57.597519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.597362    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:57.597519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.597369    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr2jq\" (UniqueName: \"kubernetes.io/projected/100dd3bb-139c-4bf3-9f3c-33e511cf9914-kube-api-access-jr2jq\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:57.597519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.597377    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/100dd3bb-139c-4bf3-9f3c-33e511cf9914-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:14:57.692476 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.692449    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"]
Sep 07 09:14:57.693748 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:14:57.693700    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39e87ab_91ed_4200_9fba_855aa30318b2.slice/crio-304f107e072d9322de70c91dba2f9dc3b7f2b9c43f3b0e8a9141cc15bd039d7e WatchSource:0}: Error finding container 304f107e072d9322de70c91dba2f9dc3b7f2b9c43f3b0e8a9141cc15bd039d7e: Status 404 returned error can't find the container with id 304f107e072d9322de70c91dba2f9dc3b7f2b9c43f3b0e8a9141cc15bd039d7e
Sep 07 09:14:57.845907 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:57.845884    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"]
Sep 07 09:14:57.847876 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:14:57.847829    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60920cf9_4df6_40e9_98dd_d54e51b848e4.slice/crio-9383a3160555528374a268a188d38ccc2f7197976a7ad3e3f873e33a9f4b6749 WatchSource:0}: Error finding container 9383a3160555528374a268a188d38ccc2f7197976a7ad3e3f873e33a9f4b6749: Status 404 returned error can't find the container with id 9383a3160555528374a268a188d38ccc2f7197976a7ad3e3f873e33a9f4b6749
Sep 07 09:14:58.073918 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.073405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr" event={"ID":"60920cf9-4df6-40e9-98dd-d54e51b848e4","Type":"ContainerStarted","Data":"dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41"}
Sep 07 09:14:58.073918 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.073470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr" event={"ID":"60920cf9-4df6-40e9-98dd-d54e51b848e4","Type":"ContainerStarted","Data":"9383a3160555528374a268a188d38ccc2f7197976a7ad3e3f873e33a9f4b6749"}
Sep 07 09:14:58.073918 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.073516    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:14:58.075155 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.075134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42" event={"ID":"f39e87ab-91ed-4200-9fba-855aa30318b2","Type":"ContainerStarted","Data":"304f107e072d9322de70c91dba2f9dc3b7f2b9c43f3b0e8a9141cc15bd039d7e"}
Sep 07 09:14:58.077348 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.077319    2563 generic.go:356] "Generic (PLEG): container finished" podID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerID="778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296" exitCode=0
Sep 07 09:14:58.077429 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.077368    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"
Sep 07 09:14:58.077471 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.077365    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" event={"ID":"100dd3bb-139c-4bf3-9f3c-33e511cf9914","Type":"ContainerDied","Data":"778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296"}
Sep 07 09:14:58.077519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.077499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx" event={"ID":"100dd3bb-139c-4bf3-9f3c-33e511cf9914","Type":"ContainerDied","Data":"69a41ed11d05b7e16780a501fec6b4fb892b683123b2521b9c294bd131085835"}
Sep 07 09:14:58.077578 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.077533    2563 scope.go:117] "RemoveContainer" containerID="778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296"
Sep 07 09:14:58.089294 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.089235    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr" podStartSLOduration=1.089217771 podStartE2EDuration="1.089217771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:58.088425666 +0000 UTC m=+956.057596272" watchObservedRunningTime="2026-09-07 09:14:58.089217771 +0000 UTC m=+956.058388376"
Sep 07 09:14:58.089519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.089502    2563 scope.go:117] "RemoveContainer" containerID="36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec"
Sep 07 09:14:58.102133 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.102111    2563 scope.go:117] "RemoveContainer" containerID="778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296"
Sep 07 09:14:58.102456 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:14:58.102434    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296\": container with ID starting with 778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296 not found: ID does not exist" containerID="778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296"
Sep 07 09:14:58.102541 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.102464    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296"} err="failed to get container status \"778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296\": rpc error: code = NotFound desc = could not find container \"778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296\": container with ID starting with 778cada26b90472227a85c1b6f82abf639572627a15eb62cbcb42ec348517296 not found: ID does not exist"
Sep 07 09:14:58.102541 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.102484    2563 scope.go:117] "RemoveContainer" containerID="36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec"
Sep 07 09:14:58.102745 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:14:58.102729    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec\": container with ID starting with 36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec not found: ID does not exist" containerID="36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec"
Sep 07 09:14:58.102789 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.102749    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec"} err="failed to get container status \"36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec\": rpc error: code = NotFound desc = could not find container \"36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec\": container with ID starting with 36c93b98814928690a66695900eda212af03e395fbe7415fd17ae2e603bcd2ec not found: ID does not exist"
Sep 07 09:14:58.120216 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.120190    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"]
Sep 07 09:14:58.124614 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.124595    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dc11d6be-openshift-default-5c9f659fb8-r94rx"]
Sep 07 09:14:58.466562 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:58.466455    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" path="/var/lib/kubelet/pods/100dd3bb-139c-4bf3-9f3c-33e511cf9914/volumes"
Sep 07 09:14:59.084097 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:59.084061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42" event={"ID":"f39e87ab-91ed-4200-9fba-855aa30318b2","Type":"ContainerStarted","Data":"600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00"}
Sep 07 09:14:59.084556 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:59.084138    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:14:59.101060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:14:59.101012    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42" podStartSLOduration=2.727950944 podStartE2EDuration="3.100996912s" totalImagesPullingTime="373.045968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:59.100475241 +0000 UTC m=+957.069645849" watchObservedRunningTime="2026-09-07 09:14:59.100996912 +0000 UTC m=+957.070167517"
Sep 07 09:15:05.098928 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:05.098892    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:15:09.087836 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:09.087797    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:15:14.170072 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:14.169972    2563 generic.go:356] "Generic (PLEG): container finished" podID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerID="1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca" exitCode=0
Sep 07 09:15:14.170072 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:14.170038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" event={"ID":"bb00b28d-deb7-4809-a3e7-d81a108ad57e","Type":"ContainerDied","Data":"1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca"}
Sep 07 09:15:14.170665 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:14.170604    2563 scope.go:117] "RemoveContainer" containerID="1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca"
Sep 07 09:15:14.171793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:14.171758    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:15:14.171896 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:14.171835    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:15:14.171896 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:14.171878    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:15:15.177637 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:15.177599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" event={"ID":"bb00b28d-deb7-4809-a3e7-d81a108ad57e","Type":"ContainerStarted","Data":"a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209"}
Sep 07 09:15:15.192341 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:15.192303    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:15:15.192631 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:15.192597    2563 patch_prober.go:28] interesting pod/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 09:15:15.192711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:15.192692    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 09:15:16.193266 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:16.193230    2563 patch_prober.go:28] interesting pod/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 09:15:16.193657 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:16.193312    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 09:15:17.192830 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:17.192794    2563 patch_prober.go:28] interesting pod/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 09:15:17.193001 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:17.192854    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 09:15:18.193397 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:18.193353    2563 patch_prober.go:28] interesting pod/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 09:15:18.193884 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:18.193410    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 09:15:19.192900 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:19.192858    2563 patch_prober.go:28] interesting pod/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 09:15:19.193117 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:19.192928    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 09:15:20.192945 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:20.192895    2563 patch_prober.go:28] interesting pod/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 09:15:20.193374 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:20.192963    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 09:15:21.196583 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:21.196546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:15:21.197026 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:21.196861    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:15:21.197535 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:21.197518    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:15:27.234443 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.234336    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"]
Sep 07 09:15:27.234999 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.234610    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42" podUID="f39e87ab-91ed-4200-9fba-855aa30318b2" containerName="maas-api" containerID="cri-o://600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00" gracePeriod=30
Sep 07 09:15:27.303197 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.303162    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"]
Sep 07 09:15:27.303413 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.303370    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr" podUID="60920cf9-4df6-40e9-98dd-d54e51b848e4" containerName="payload-pre-processing" containerID="cri-o://dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41" gracePeriod=30
Sep 07 09:15:27.568401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.568371    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"]
Sep 07 09:15:27.568711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.568683    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy" containerID="cri-o://a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209" gracePeriod=30
Sep 07 09:15:27.582259 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.582230    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:15:27.609303 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.609257    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:15:27.694904 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.694864    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29sz8\" (UniqueName: \"kubernetes.io/projected/60920cf9-4df6-40e9-98dd-d54e51b848e4-kube-api-access-29sz8\") pod \"60920cf9-4df6-40e9-98dd-d54e51b848e4\" (UID: \"60920cf9-4df6-40e9-98dd-d54e51b848e4\") "
Sep 07 09:15:27.695103 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.694964    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-maas-api-tls\") pod \"f39e87ab-91ed-4200-9fba-855aa30318b2\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") "
Sep 07 09:15:27.695103 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.694988    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60920cf9-4df6-40e9-98dd-d54e51b848e4-plugins-config-volume\") pod \"60920cf9-4df6-40e9-98dd-d54e51b848e4\" (UID: \"60920cf9-4df6-40e9-98dd-d54e51b848e4\") "
Sep 07 09:15:27.695103 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.695008    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39e87ab-91ed-4200-9fba-855aa30318b2-openshift-service-ca\") pod \"f39e87ab-91ed-4200-9fba-855aa30318b2\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") "
Sep 07 09:15:27.695103 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.695052    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr7zf\" (UniqueName: \"kubernetes.io/projected/f39e87ab-91ed-4200-9fba-855aa30318b2-kube-api-access-dr7zf\") pod \"f39e87ab-91ed-4200-9fba-855aa30318b2\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") "
Sep 07 09:15:27.695358 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.695109    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-metrics-certs\") pod \"f39e87ab-91ed-4200-9fba-855aa30318b2\" (UID: \"f39e87ab-91ed-4200-9fba-855aa30318b2\") "
Sep 07 09:15:27.695493 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.695465    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39e87ab-91ed-4200-9fba-855aa30318b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f39e87ab-91ed-4200-9fba-855aa30318b2" (UID: "f39e87ab-91ed-4200-9fba-855aa30318b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:15:27.695587 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.695499    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60920cf9-4df6-40e9-98dd-d54e51b848e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "60920cf9-4df6-40e9-98dd-d54e51b848e4" (UID: "60920cf9-4df6-40e9-98dd-d54e51b848e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:15:27.697383 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.697350    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f39e87ab-91ed-4200-9fba-855aa30318b2" (UID: "f39e87ab-91ed-4200-9fba-855aa30318b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:15:27.697508 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.697478    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f39e87ab-91ed-4200-9fba-855aa30318b2" (UID: "f39e87ab-91ed-4200-9fba-855aa30318b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:15:27.697695 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.697670    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39e87ab-91ed-4200-9fba-855aa30318b2-kube-api-access-dr7zf" (OuterVolumeSpecName: "kube-api-access-dr7zf") pod "f39e87ab-91ed-4200-9fba-855aa30318b2" (UID: "f39e87ab-91ed-4200-9fba-855aa30318b2"). InnerVolumeSpecName "kube-api-access-dr7zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:15:27.697741 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.697709    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60920cf9-4df6-40e9-98dd-d54e51b848e4-kube-api-access-29sz8" (OuterVolumeSpecName: "kube-api-access-29sz8") pod "60920cf9-4df6-40e9-98dd-d54e51b848e4" (UID: "60920cf9-4df6-40e9-98dd-d54e51b848e4"). InnerVolumeSpecName "kube-api-access-29sz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:15:27.796795 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.796713    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:27.796795 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.796741    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60920cf9-4df6-40e9-98dd-d54e51b848e4-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:27.796795 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.796752    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39e87ab-91ed-4200-9fba-855aa30318b2-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:27.796795 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.796760    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr7zf\" (UniqueName: \"kubernetes.io/projected/f39e87ab-91ed-4200-9fba-855aa30318b2-kube-api-access-dr7zf\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:27.796795 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.796769    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f39e87ab-91ed-4200-9fba-855aa30318b2-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:27.796795 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:27.796777    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29sz8\" (UniqueName: \"kubernetes.io/projected/60920cf9-4df6-40e9-98dd-d54e51b848e4-kube-api-access-29sz8\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:28.253275 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.253240    2563 generic.go:356] "Generic (PLEG): container finished" podID="f39e87ab-91ed-4200-9fba-855aa30318b2" containerID="600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00" exitCode=0
Sep 07 09:15:28.253712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.253318    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"
Sep 07 09:15:28.253712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.253330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42" event={"ID":"f39e87ab-91ed-4200-9fba-855aa30318b2","Type":"ContainerDied","Data":"600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00"}
Sep 07 09:15:28.253712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.253402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42" event={"ID":"f39e87ab-91ed-4200-9fba-855aa30318b2","Type":"ContainerDied","Data":"304f107e072d9322de70c91dba2f9dc3b7f2b9c43f3b0e8a9141cc15bd039d7e"}
Sep 07 09:15:28.253712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.253427    2563 scope.go:117] "RemoveContainer" containerID="600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00"
Sep 07 09:15:28.254916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.254831    2563 generic.go:356] "Generic (PLEG): container finished" podID="60920cf9-4df6-40e9-98dd-d54e51b848e4" containerID="dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41" exitCode=0
Sep 07 09:15:28.254916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.254865    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"
Sep 07 09:15:28.254916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.254907    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr" event={"ID":"60920cf9-4df6-40e9-98dd-d54e51b848e4","Type":"ContainerDied","Data":"dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41"}
Sep 07 09:15:28.255077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.254948    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr" event={"ID":"60920cf9-4df6-40e9-98dd-d54e51b848e4","Type":"ContainerDied","Data":"9383a3160555528374a268a188d38ccc2f7197976a7ad3e3f873e33a9f4b6749"}
Sep 07 09:15:28.266196 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.266179    2563 scope.go:117] "RemoveContainer" containerID="600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00"
Sep 07 09:15:28.266485 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:15:28.266466    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00\": container with ID starting with 600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00 not found: ID does not exist" containerID="600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00"
Sep 07 09:15:28.266576 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.266495    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00"} err="failed to get container status \"600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00\": rpc error: code = NotFound desc = could not find container \"600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00\": container with ID starting with 600157591c93ebe77cc767743271c625378ef29da8674ffc61efe59371c3ce00 not found: ID does not exist"
Sep 07 09:15:28.266576 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.266517    2563 scope.go:117] "RemoveContainer" containerID="dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41"
Sep 07 09:15:28.274057 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.274040    2563 scope.go:117] "RemoveContainer" containerID="dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41"
Sep 07 09:15:28.274322 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:15:28.274305    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41\": container with ID starting with dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41 not found: ID does not exist" containerID="dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41"
Sep 07 09:15:28.274388 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.274327    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41"} err="failed to get container status \"dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41\": rpc error: code = NotFound desc = could not find container \"dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41\": container with ID starting with dbbf4d64ef71460fc498f325e3db9054353e7b3202d7ecaeef5e45c8b1b67e41 not found: ID does not exist"
Sep 07 09:15:28.278799 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.278778    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"]
Sep 07 09:15:28.283806 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.283788    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-863fd8cf-58fb9ddd79-2jj42"]
Sep 07 09:15:28.294435 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.294415    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"]
Sep 07 09:15:28.300959 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.300941    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-863fd8cf-c457fb78b-dtfxr"]
Sep 07 09:15:28.466628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.466595    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60920cf9-4df6-40e9-98dd-d54e51b848e4" path="/var/lib/kubelet/pods/60920cf9-4df6-40e9-98dd-d54e51b848e4/volumes"
Sep 07 09:15:28.467010 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:28.466993    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f39e87ab-91ed-4200-9fba-855aa30318b2" path="/var/lib/kubelet/pods/f39e87ab-91ed-4200-9fba-855aa30318b2/volumes"
Sep 07 09:15:32.825235 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.825209    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:15:32.948031 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.947934    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-data\") pod \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") "
Sep 07 09:15:32.948031 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.947971    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istiod-ca-cert\") pod \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") "
Sep 07 09:15:32.948031 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948000    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-certs\") pod \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") "
Sep 07 09:15:32.948351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948036    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xx9h\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-kube-api-access-7xx9h\") pod \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") "
Sep 07 09:15:32.948351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948069    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-podinfo\") pod \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") "
Sep 07 09:15:32.948351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948104    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-envoy\") pod \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") "
Sep 07 09:15:32.948351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948171    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-credential-socket\") pod \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") "
Sep 07 09:15:32.948351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948201    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-token\") pod \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") "
Sep 07 09:15:32.948351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948258    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-socket\") pod \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\" (UID: \"bb00b28d-deb7-4809-a3e7-d81a108ad57e\") "
Sep 07 09:15:32.948351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948271    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb00b28d-deb7-4809-a3e7-d81a108ad57e" (UID: "bb00b28d-deb7-4809-a3e7-d81a108ad57e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:32.948732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948472    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb00b28d-deb7-4809-a3e7-d81a108ad57e" (UID: "bb00b28d-deb7-4809-a3e7-d81a108ad57e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:32.948732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948489    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb00b28d-deb7-4809-a3e7-d81a108ad57e" (UID: "bb00b28d-deb7-4809-a3e7-d81a108ad57e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:15:32.948732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948525    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:32.948732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948724    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb00b28d-deb7-4809-a3e7-d81a108ad57e" (UID: "bb00b28d-deb7-4809-a3e7-d81a108ad57e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:32.948956 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.948941    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb00b28d-deb7-4809-a3e7-d81a108ad57e" (UID: "bb00b28d-deb7-4809-a3e7-d81a108ad57e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:32.950466 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.950432    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb00b28d-deb7-4809-a3e7-d81a108ad57e" (UID: "bb00b28d-deb7-4809-a3e7-d81a108ad57e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:32.950582 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.950515    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb00b28d-deb7-4809-a3e7-d81a108ad57e" (UID: "bb00b28d-deb7-4809-a3e7-d81a108ad57e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:15:32.950582 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.950548    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-kube-api-access-7xx9h" (OuterVolumeSpecName: "kube-api-access-7xx9h") pod "bb00b28d-deb7-4809-a3e7-d81a108ad57e" (UID: "bb00b28d-deb7-4809-a3e7-d81a108ad57e"). InnerVolumeSpecName "kube-api-access-7xx9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:15:32.950993 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:32.950976    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb00b28d-deb7-4809-a3e7-d81a108ad57e" (UID: "bb00b28d-deb7-4809-a3e7-d81a108ad57e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:15:33.049456 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.049418    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:33.049456 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.049448    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:33.049456 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.049461    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:33.049676 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.049469    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:33.049676 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.049478    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xx9h\" (UniqueName: \"kubernetes.io/projected/bb00b28d-deb7-4809-a3e7-d81a108ad57e-kube-api-access-7xx9h\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:33.049676 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.049486    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:33.049676 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.049493    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:33.049676 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.049503    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb00b28d-deb7-4809-a3e7-d81a108ad57e-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:33.283778 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.283735    2563 generic.go:356] "Generic (PLEG): container finished" podID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerID="a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209" exitCode=0
Sep 07 09:15:33.283971 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.283787    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"
Sep 07 09:15:33.283971 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.283820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" event={"ID":"bb00b28d-deb7-4809-a3e7-d81a108ad57e","Type":"ContainerDied","Data":"a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209"}
Sep 07 09:15:33.283971 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.283890    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5" event={"ID":"bb00b28d-deb7-4809-a3e7-d81a108ad57e","Type":"ContainerDied","Data":"f1ebb7e911743858f75109967e7cc3d1b0cf08f93a6e6d99a38ab1721010b55a"}
Sep 07 09:15:33.283971 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.283916    2563 scope.go:117] "RemoveContainer" containerID="a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209"
Sep 07 09:15:33.293414 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.293395    2563 scope.go:117] "RemoveContainer" containerID="1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca"
Sep 07 09:15:33.301502 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.301483    2563 scope.go:117] "RemoveContainer" containerID="a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209"
Sep 07 09:15:33.301785 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:15:33.301764    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209\": container with ID starting with a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209 not found: ID does not exist" containerID="a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209"
Sep 07 09:15:33.301847 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.301795    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209"} err="failed to get container status \"a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209\": rpc error: code = NotFound desc = could not find container \"a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209\": container with ID starting with a41aaed1b1208752b0834fbbf164237edf73072d801df2e72c170c25266c2209 not found: ID does not exist"
Sep 07 09:15:33.301847 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.301813    2563 scope.go:117] "RemoveContainer" containerID="1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca"
Sep 07 09:15:33.302078 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:15:33.302051    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca\": container with ID starting with 1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca not found: ID does not exist" containerID="1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca"
Sep 07 09:15:33.302131 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.302086    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca"} err="failed to get container status \"1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca\": rpc error: code = NotFound desc = could not find container \"1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca\": container with ID starting with 1b81e6d3bfe08dc383395bbb729915ea9f4ea90147b4f356e2b033a2417c2aca not found: ID does not exist"
Sep 07 09:15:33.305203 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.305182    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"]
Sep 07 09:15:33.308823 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:33.308799    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-863fd8cf-openshift-default-69f98f6dc6-g7nz5"]
Sep 07 09:15:34.465337 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:34.465299    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" path="/var/lib/kubelet/pods/bb00b28d-deb7-4809-a3e7-d81a108ad57e/volumes"
Sep 07 09:15:35.230600 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:35.230566    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"]
Sep 07 09:15:35.230798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:35.230777    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6" podUID="1497cacf-a480-4296-b250-afb66ba5d1ed" containerName="payload-processing" containerID="cri-o://74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f" gracePeriod=30
Sep 07 09:15:35.490122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:35.490059    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:15:35.575305 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:35.575252    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1497cacf-a480-4296-b250-afb66ba5d1ed-plugins-config-volume\") pod \"1497cacf-a480-4296-b250-afb66ba5d1ed\" (UID: \"1497cacf-a480-4296-b250-afb66ba5d1ed\") "
Sep 07 09:15:35.575561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:35.575333    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx52x\" (UniqueName: \"kubernetes.io/projected/1497cacf-a480-4296-b250-afb66ba5d1ed-kube-api-access-wx52x\") pod \"1497cacf-a480-4296-b250-afb66ba5d1ed\" (UID: \"1497cacf-a480-4296-b250-afb66ba5d1ed\") "
Sep 07 09:15:35.576091 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:35.575868    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1497cacf-a480-4296-b250-afb66ba5d1ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1497cacf-a480-4296-b250-afb66ba5d1ed" (UID: "1497cacf-a480-4296-b250-afb66ba5d1ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:15:35.576091 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:35.576020    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1497cacf-a480-4296-b250-afb66ba5d1ed-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:35.578076 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:35.578045    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1497cacf-a480-4296-b250-afb66ba5d1ed-kube-api-access-wx52x" (OuterVolumeSpecName: "kube-api-access-wx52x") pod "1497cacf-a480-4296-b250-afb66ba5d1ed" (UID: "1497cacf-a480-4296-b250-afb66ba5d1ed"). InnerVolumeSpecName "kube-api-access-wx52x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:15:35.677489 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:35.677447    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx52x\" (UniqueName: \"kubernetes.io/projected/1497cacf-a480-4296-b250-afb66ba5d1ed-kube-api-access-wx52x\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:15:36.307636 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.307599    2563 generic.go:356] "Generic (PLEG): container finished" podID="1497cacf-a480-4296-b250-afb66ba5d1ed" containerID="74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f" exitCode=0
Sep 07 09:15:36.307826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.307650    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"
Sep 07 09:15:36.307826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.307671    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6" event={"ID":"1497cacf-a480-4296-b250-afb66ba5d1ed","Type":"ContainerDied","Data":"74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f"}
Sep 07 09:15:36.307826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.307715    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6" event={"ID":"1497cacf-a480-4296-b250-afb66ba5d1ed","Type":"ContainerDied","Data":"c4048c0c1aaf23e7389a501752312dd4bdc6be86a7cce131ca4bfbb169b68fe0"}
Sep 07 09:15:36.307826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.307736    2563 scope.go:117] "RemoveContainer" containerID="74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f"
Sep 07 09:15:36.317214 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.317194    2563 scope.go:117] "RemoveContainer" containerID="74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f"
Sep 07 09:15:36.317542 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:15:36.317519    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f\": container with ID starting with 74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f not found: ID does not exist" containerID="74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f"
Sep 07 09:15:36.317616 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.317554    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f"} err="failed to get container status \"74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f\": rpc error: code = NotFound desc = could not find container \"74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f\": container with ID starting with 74200141f84c11f461581a86a86d127f4a65f98b39e61dad23b18cbeff58a68f not found: ID does not exist"
Sep 07 09:15:36.330828 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.330801    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"]
Sep 07 09:15:36.334522 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.334496    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-da4d4c-57b8598845-p27v6"]
Sep 07 09:15:36.464383 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:36.464351    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1497cacf-a480-4296-b250-afb66ba5d1ed" path="/var/lib/kubelet/pods/1497cacf-a480-4296-b250-afb66ba5d1ed/volumes"
Sep 07 09:15:42.590972 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.590941    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"]
Sep 07 09:15:42.591401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591343    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1497cacf-a480-4296-b250-afb66ba5d1ed" containerName="payload-processing"
Sep 07 09:15:42.591401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591355    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1497cacf-a480-4296-b250-afb66ba5d1ed" containerName="payload-processing"
Sep 07 09:15:42.591401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591378    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39e87ab-91ed-4200-9fba-855aa30318b2" containerName="maas-api"
Sep 07 09:15:42.591401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591385    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39e87ab-91ed-4200-9fba-855aa30318b2" containerName="maas-api"
Sep 07 09:15:42.591401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591391    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy"
Sep 07 09:15:42.591401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591396    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy"
Sep 07 09:15:42.591401 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591405    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591411    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591417    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591422    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591434    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60920cf9-4df6-40e9-98dd-d54e51b848e4" containerName="payload-pre-processing"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591439    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="60920cf9-4df6-40e9-98dd-d54e51b848e4" containerName="payload-pre-processing"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591491    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591499    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591506    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39e87ab-91ed-4200-9fba-855aa30318b2" containerName="maas-api"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591511    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="60920cf9-4df6-40e9-98dd-d54e51b848e4" containerName="payload-pre-processing"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591518    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591524    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="100dd3bb-139c-4bf3-9f3c-33e511cf9914" containerName="istio-proxy"
Sep 07 09:15:42.591628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.591531    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1497cacf-a480-4296-b250-afb66ba5d1ed" containerName="payload-processing"
Sep 07 09:15:42.595677 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.595659    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.598441 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.598420    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-085b77-openshift-default-dockercfg-47mn2\""
Sep 07 09:15:42.611551 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.611526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"]
Sep 07 09:15:42.746816 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.746779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-728bg\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-kube-api-access-728bg\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.746992 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.746828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-token\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.746992 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.746873    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istiod-ca-cert\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.746992 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.746932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-data\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.747127 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.746992    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-envoy\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.747127 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.747025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-podinfo\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.747127 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.747059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-socket\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.747127 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.747087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-certs\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.747127 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.747120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-credential-socket\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848336 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-728bg\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-kube-api-access-728bg\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848336 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848274    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-token\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848336 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848335    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istiod-ca-cert\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848638 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-data\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848638 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-envoy\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848638 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-podinfo\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848638 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-socket\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848854 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-certs\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848854 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-credential-socket\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.848854 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-data\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.849013 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.848922    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-socket\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.849057 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.849031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-certs\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.849095 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.849077    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-credential-socket\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.849392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.849374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istiod-ca-cert\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.850938 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.850915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-envoy\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.851058 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.851041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-podinfo\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.856232 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.856207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-token\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.856373 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.856354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-728bg\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-kube-api-access-728bg\") pod \"e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") " pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:42.905402 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:42.905358    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:43.189637 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:43.189523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"]
Sep 07 09:15:43.191825 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:15:43.191798    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd273cb58_8cb9_46fa_9e9c_5fd81f0a3ddc.slice/crio-be4d39ebe78b3b6361072638fca2f99270608eb89f0b1d0158f856276e3f95f7 WatchSource:0}: Error finding container be4d39ebe78b3b6361072638fca2f99270608eb89f0b1d0158f856276e3f95f7: Status 404 returned error can't find the container with id be4d39ebe78b3b6361072638fca2f99270608eb89f0b1d0158f856276e3f95f7
Sep 07 09:15:43.194172 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:43.194137    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:15:43.194229 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:43.194213    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:15:43.194267 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:43.194248    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:15:43.346671 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:43.346633    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" event={"ID":"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc","Type":"ContainerStarted","Data":"e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da"}
Sep 07 09:15:43.346856 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:43.346703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" event={"ID":"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc","Type":"ContainerStarted","Data":"be4d39ebe78b3b6361072638fca2f99270608eb89f0b1d0158f856276e3f95f7"}
Sep 07 09:15:43.366321 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:43.366235    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" podStartSLOduration=1.36621151 podStartE2EDuration="1.36621151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:15:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:15:43.364246142 +0000 UTC m=+1001.333416748" watchObservedRunningTime="2026-09-07 09:15:43.36621151 +0000 UTC m=+1001.335382116"
Sep 07 09:15:43.906096 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:43.906055    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:44.911177 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:44.911146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:45.357682 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:45.357651    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:45.358565 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:45.358538    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:15:46.502451 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.502415    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"]
Sep 07 09:15:46.502902 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.502886    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy"
Sep 07 09:15:46.502963 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.502905    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb00b28d-deb7-4809-a3e7-d81a108ad57e" containerName="istio-proxy"
Sep 07 09:15:46.506441 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.506414    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.511211 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.511191    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-085b77-serving-cert\""
Sep 07 09:15:46.520139 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.520107    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"]
Sep 07 09:15:46.687121 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.687083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpt8s\" (UniqueName: \"kubernetes.io/projected/edd6761d-5ede-4063-a9da-58c93f700d8b-kube-api-access-rpt8s\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.687344 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.687146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-maas-api-tls\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.687344 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.687204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-metrics-certs\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.687344 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.687309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd6761d-5ede-4063-a9da-58c93f700d8b-openshift-service-ca\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.788104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.788001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-maas-api-tls\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.788104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.788055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-metrics-certs\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.788403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.788138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd6761d-5ede-4063-a9da-58c93f700d8b-openshift-service-ca\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.788403 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:15:46.788164    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-serving-cert: secret "maas-api-e2e-explicit-085b77-serving-cert" not found
Sep 07 09:15:46.788403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.788203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpt8s\" (UniqueName: \"kubernetes.io/projected/edd6761d-5ede-4063-a9da-58c93f700d8b-kube-api-access-rpt8s\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.788403 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:15:46.788233    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-maas-api-tls podName:edd6761d-5ede-4063-a9da-58c93f700d8b nodeName:}" failed. No retries permitted until 2026-09-07 09:15:47.288216107 +0000 UTC m=+1005.257386714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-maas-api-tls") pod "maas-api-e2e-explicit-085b77-585cc5cf97-78cdl" (UID: "edd6761d-5ede-4063-a9da-58c93f700d8b") : secret "maas-api-e2e-explicit-085b77-serving-cert" not found
Sep 07 09:15:46.788916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.788898    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd6761d-5ede-4063-a9da-58c93f700d8b-openshift-service-ca\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.790536 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.790518    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-metrics-certs\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:46.797781 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:46.797756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpt8s\" (UniqueName: \"kubernetes.io/projected/edd6761d-5ede-4063-a9da-58c93f700d8b-kube-api-access-rpt8s\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:47.111462 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.111368    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"]
Sep 07 09:15:47.115146 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.115123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:47.118781 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.118757    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-085b77-dockercfg-7p4d4\""
Sep 07 09:15:47.118902 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.118763    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-085b77\""
Sep 07 09:15:47.128779 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.128755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"]
Sep 07 09:15:47.146480 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.146449    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"]
Sep 07 09:15:47.150692 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.150664    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:15:47.160877 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.160849    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"]
Sep 07 09:15:47.293719 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.293679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e10f13-238f-4cf6-8602-8ed114eb6d1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk\" (UID: \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:47.293918 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.293795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf45b\" (UniqueName: \"kubernetes.io/projected/39f4400b-1923-44d5-b955-caf2c0b6b445-kube-api-access-nf45b\") pod \"payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9\" (UID: \"39f4400b-1923-44d5-b955-caf2c0b6b445\") " pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:15:47.293918 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.293844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-maas-api-tls\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:47.293918 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.293875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxhc6\" (UniqueName: \"kubernetes.io/projected/56e10f13-238f-4cf6-8602-8ed114eb6d1c-kube-api-access-mxhc6\") pod \"payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk\" (UID: \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:47.294070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.293923    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f4400b-1923-44d5-b955-caf2c0b6b445-plugins-config-volume\") pod \"payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9\" (UID: \"39f4400b-1923-44d5-b955-caf2c0b6b445\") " pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:15:47.296189 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.296162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-maas-api-tls\") pod \"maas-api-e2e-explicit-085b77-585cc5cf97-78cdl\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:47.395256 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.395181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e10f13-238f-4cf6-8602-8ed114eb6d1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk\" (UID: \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:47.395460 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.395321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf45b\" (UniqueName: \"kubernetes.io/projected/39f4400b-1923-44d5-b955-caf2c0b6b445-kube-api-access-nf45b\") pod \"payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9\" (UID: \"39f4400b-1923-44d5-b955-caf2c0b6b445\") " pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:15:47.395460 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.395368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxhc6\" (UniqueName: \"kubernetes.io/projected/56e10f13-238f-4cf6-8602-8ed114eb6d1c-kube-api-access-mxhc6\") pod \"payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk\" (UID: \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:47.395460 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.395428    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f4400b-1923-44d5-b955-caf2c0b6b445-plugins-config-volume\") pod \"payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9\" (UID: \"39f4400b-1923-44d5-b955-caf2c0b6b445\") " pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:15:47.396112 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.396088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e10f13-238f-4cf6-8602-8ed114eb6d1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk\" (UID: \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:47.396202 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.396185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f4400b-1923-44d5-b955-caf2c0b6b445-plugins-config-volume\") pod \"payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9\" (UID: \"39f4400b-1923-44d5-b955-caf2c0b6b445\") " pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:15:47.404696 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.404663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxhc6\" (UniqueName: \"kubernetes.io/projected/56e10f13-238f-4cf6-8602-8ed114eb6d1c-kube-api-access-mxhc6\") pod \"payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk\" (UID: \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:47.404888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.404866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf45b\" (UniqueName: \"kubernetes.io/projected/39f4400b-1923-44d5-b955-caf2c0b6b445-kube-api-access-nf45b\") pod \"payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9\" (UID: \"39f4400b-1923-44d5-b955-caf2c0b6b445\") " pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:15:47.421722 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.421694    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:47.425722 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.425601    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:47.463245 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.463198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:15:47.710181 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.710157    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"]
Sep 07 09:15:47.711787 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:15:47.711753    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd6761d_5ede_4063_a9da_58c93f700d8b.slice/crio-fb5d0b64cbbecbc2816d7ac4855641e5d93df4d30b167c19217ed27c1b14617a WatchSource:0}: Error finding container fb5d0b64cbbecbc2816d7ac4855641e5d93df4d30b167c19217ed27c1b14617a: Status 404 returned error can't find the container with id fb5d0b64cbbecbc2816d7ac4855641e5d93df4d30b167c19217ed27c1b14617a
Sep 07 09:15:47.720980 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.720957    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"]
Sep 07 09:15:47.722477 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:15:47.722450    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e10f13_238f_4cf6_8602_8ed114eb6d1c.slice/crio-b51c3241dc1155acfaadde410661922f9902ea3d5115157c0244d406e1e31d3f WatchSource:0}: Error finding container b51c3241dc1155acfaadde410661922f9902ea3d5115157c0244d406e1e31d3f: Status 404 returned error can't find the container with id b51c3241dc1155acfaadde410661922f9902ea3d5115157c0244d406e1e31d3f
Sep 07 09:15:47.756139 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:47.756116    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"]
Sep 07 09:15:47.759208 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:15:47.759183    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39f4400b_1923_44d5_b955_caf2c0b6b445.slice/crio-631f2b93fe4a0c4a20520319b65a641f3501579919c3274f7f3dc586a60700d2 WatchSource:0}: Error finding container 631f2b93fe4a0c4a20520319b65a641f3501579919c3274f7f3dc586a60700d2: Status 404 returned error can't find the container with id 631f2b93fe4a0c4a20520319b65a641f3501579919c3274f7f3dc586a60700d2
Sep 07 09:15:48.375537 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.375437    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9" event={"ID":"39f4400b-1923-44d5-b955-caf2c0b6b445","Type":"ContainerStarted","Data":"b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584"}
Sep 07 09:15:48.375537 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.375493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9" event={"ID":"39f4400b-1923-44d5-b955-caf2c0b6b445","Type":"ContainerStarted","Data":"631f2b93fe4a0c4a20520319b65a641f3501579919c3274f7f3dc586a60700d2"}
Sep 07 09:15:48.375537 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.375527    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:15:48.377134 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.377088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk" event={"ID":"56e10f13-238f-4cf6-8602-8ed114eb6d1c","Type":"ContainerStarted","Data":"ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a"}
Sep 07 09:15:48.377272 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.377154    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:48.377272 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.377181    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk" event={"ID":"56e10f13-238f-4cf6-8602-8ed114eb6d1c","Type":"ContainerStarted","Data":"b51c3241dc1155acfaadde410661922f9902ea3d5115157c0244d406e1e31d3f"}
Sep 07 09:15:48.378749 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.378726    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl" event={"ID":"edd6761d-5ede-4063-a9da-58c93f700d8b","Type":"ContainerStarted","Data":"5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03"}
Sep 07 09:15:48.378873 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.378776    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:48.378873 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.378792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl" event={"ID":"edd6761d-5ede-4063-a9da-58c93f700d8b","Type":"ContainerStarted","Data":"fb5d0b64cbbecbc2816d7ac4855641e5d93df4d30b167c19217ed27c1b14617a"}
Sep 07 09:15:48.394138 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.394085    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9" podStartSLOduration=1.394067809 podStartE2EDuration="1.394067809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:15:48.392601344 +0000 UTC m=+1006.361771950" watchObservedRunningTime="2026-09-07 09:15:48.394067809 +0000 UTC m=+1006.363238477"
Sep 07 09:15:48.411443 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.411388    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk" podStartSLOduration=1.411370635 podStartE2EDuration="1.411370635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:15:48.41077664 +0000 UTC m=+1006.379947247" watchObservedRunningTime="2026-09-07 09:15:48.411370635 +0000 UTC m=+1006.380541240"
Sep 07 09:15:48.437973 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:48.437921    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl" podStartSLOduration=2.047697762 podStartE2EDuration="2.437902434s" totalImagesPullingTime="390.204672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:15:48.436076976 +0000 UTC m=+1006.405247583" watchObservedRunningTime="2026-09-07 09:15:48.437902434 +0000 UTC m=+1006.407073039"
Sep 07 09:15:54.392170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:54.392132    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:15:59.388406 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:59.388352    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:15:59.388786 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:15:59.388440    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:16:01.399533 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:01.399485    2563 patch_prober.go:28] interesting pod/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 09:16:01.399952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:01.399562    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 09:16:03.465143 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:03.465108    2563 generic.go:356] "Generic (PLEG): container finished" podID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerID="e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da" exitCode=0
Sep 07 09:16:03.465693 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:03.465187    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" event={"ID":"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc","Type":"ContainerDied","Data":"e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da"}
Sep 07 09:16:03.465786 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:03.465692    2563 scope.go:117] "RemoveContainer" containerID="e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da"
Sep 07 09:16:03.466777 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:03.466751    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:16:03.466844 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:03.466819    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:16:03.466885 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:03.466849    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:16:04.474197 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:04.474164    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" event={"ID":"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc","Type":"ContainerStarted","Data":"5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0"}
Sep 07 09:16:04.906437 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:04.906412    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:16:04.906649 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:04.906624    2563 patch_prober.go:28] interesting pod/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 07 09:16:04.906723 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:04.906676    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 07 09:16:05.906121 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:05.906079    2563 patch_prober.go:28] interesting pod/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 07 09:16:05.906529 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:05.906140    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 07 09:16:06.906191 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:06.906153    2563 patch_prober.go:28] interesting pod/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 07 09:16:06.906579 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:06.906208    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 07 09:16:07.906730 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:07.906688    2563 patch_prober.go:28] interesting pod/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 07 09:16:07.907223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:07.906749    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 07 09:16:08.906802 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:08.906765    2563 patch_prober.go:28] interesting pod/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 07 09:16:08.907196 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:08.906824    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 07 09:16:09.910129 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:09.910095    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:16:09.910626 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:09.910506    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:16:09.911158 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:16:09.911136    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:17:02.115104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.115007    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"]
Sep 07 09:17:02.115638 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.115319    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl" podUID="edd6761d-5ede-4063-a9da-58c93f700d8b" containerName="maas-api" containerID="cri-o://5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03" gracePeriod=30
Sep 07 09:17:02.181360 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.181320    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"]
Sep 07 09:17:02.181532 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.181506    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9" podUID="39f4400b-1923-44d5-b955-caf2c0b6b445" containerName="payload-processing" containerID="cri-o://b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584" gracePeriod=30
Sep 07 09:17:02.210847 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.210806    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"]
Sep 07 09:17:02.211040 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.211018    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk" podUID="56e10f13-238f-4cf6-8602-8ed114eb6d1c" containerName="payload-pre-processing" containerID="cri-o://ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a" gracePeriod=30
Sep 07 09:17:02.463887 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.463833    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:17:02.498356 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.498275    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:17:02.502739 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.502716    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:17:02.545124 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.545096    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd6761d-5ede-4063-a9da-58c93f700d8b-openshift-service-ca\") pod \"edd6761d-5ede-4063-a9da-58c93f700d8b\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") "
Sep 07 09:17:02.545330 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.545140    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpt8s\" (UniqueName: \"kubernetes.io/projected/edd6761d-5ede-4063-a9da-58c93f700d8b-kube-api-access-rpt8s\") pod \"edd6761d-5ede-4063-a9da-58c93f700d8b\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") "
Sep 07 09:17:02.545330 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.545192    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-metrics-certs\") pod \"edd6761d-5ede-4063-a9da-58c93f700d8b\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") "
Sep 07 09:17:02.545330 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.545239    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-maas-api-tls\") pod \"edd6761d-5ede-4063-a9da-58c93f700d8b\" (UID: \"edd6761d-5ede-4063-a9da-58c93f700d8b\") "
Sep 07 09:17:02.545547 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.545508    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd6761d-5ede-4063-a9da-58c93f700d8b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "edd6761d-5ede-4063-a9da-58c93f700d8b" (UID: "edd6761d-5ede-4063-a9da-58c93f700d8b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:17:02.545636 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.545612    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd6761d-5ede-4063-a9da-58c93f700d8b-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:02.547463 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.547437    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "edd6761d-5ede-4063-a9da-58c93f700d8b" (UID: "edd6761d-5ede-4063-a9da-58c93f700d8b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:17:02.547574 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.547466    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "edd6761d-5ede-4063-a9da-58c93f700d8b" (UID: "edd6761d-5ede-4063-a9da-58c93f700d8b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:17:02.547746 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.547730    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd6761d-5ede-4063-a9da-58c93f700d8b-kube-api-access-rpt8s" (OuterVolumeSpecName: "kube-api-access-rpt8s") pod "edd6761d-5ede-4063-a9da-58c93f700d8b" (UID: "edd6761d-5ede-4063-a9da-58c93f700d8b"). InnerVolumeSpecName "kube-api-access-rpt8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:02.587728 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.587698    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"]
Sep 07 09:17:02.588126 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.588113    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e10f13-238f-4cf6-8602-8ed114eb6d1c" containerName="payload-pre-processing"
Sep 07 09:17:02.588189 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.588130    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e10f13-238f-4cf6-8602-8ed114eb6d1c" containerName="payload-pre-processing"
Sep 07 09:17:02.588189 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.588143    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edd6761d-5ede-4063-a9da-58c93f700d8b" containerName="maas-api"
Sep 07 09:17:02.588189 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.588149    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="edd6761d-5ede-4063-a9da-58c93f700d8b" containerName="maas-api"
Sep 07 09:17:02.588189 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.588164    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39f4400b-1923-44d5-b955-caf2c0b6b445" containerName="payload-processing"
Sep 07 09:17:02.588189 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.588170    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="39f4400b-1923-44d5-b955-caf2c0b6b445" containerName="payload-processing"
Sep 07 09:17:02.588381 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.588230    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="edd6761d-5ede-4063-a9da-58c93f700d8b" containerName="maas-api"
Sep 07 09:17:02.588381 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.588240    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="39f4400b-1923-44d5-b955-caf2c0b6b445" containerName="payload-processing"
Sep 07 09:17:02.588381 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.588247    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e10f13-238f-4cf6-8602-8ed114eb6d1c" containerName="payload-pre-processing"
Sep 07 09:17:02.591372 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.591350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.593859 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.593837    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e8e5dd53-openshift-default-dockercfg-2rdbg\""
Sep 07 09:17:02.606570 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.606543    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"]
Sep 07 09:17:02.646098 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.646009    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxhc6\" (UniqueName: \"kubernetes.io/projected/56e10f13-238f-4cf6-8602-8ed114eb6d1c-kube-api-access-mxhc6\") pod \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\" (UID: \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\") "
Sep 07 09:17:02.646098 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.646066    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf45b\" (UniqueName: \"kubernetes.io/projected/39f4400b-1923-44d5-b955-caf2c0b6b445-kube-api-access-nf45b\") pod \"39f4400b-1923-44d5-b955-caf2c0b6b445\" (UID: \"39f4400b-1923-44d5-b955-caf2c0b6b445\") "
Sep 07 09:17:02.646348 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.646152    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f4400b-1923-44d5-b955-caf2c0b6b445-plugins-config-volume\") pod \"39f4400b-1923-44d5-b955-caf2c0b6b445\" (UID: \"39f4400b-1923-44d5-b955-caf2c0b6b445\") "
Sep 07 09:17:02.646348 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.646250    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e10f13-238f-4cf6-8602-8ed114eb6d1c-plugins-config-volume\") pod \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\" (UID: \"56e10f13-238f-4cf6-8602-8ed114eb6d1c\") "
Sep 07 09:17:02.646628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.646598    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39f4400b-1923-44d5-b955-caf2c0b6b445-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39f4400b-1923-44d5-b955-caf2c0b6b445" (UID: "39f4400b-1923-44d5-b955-caf2c0b6b445"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:17:02.646628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.646624    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpt8s\" (UniqueName: \"kubernetes.io/projected/edd6761d-5ede-4063-a9da-58c93f700d8b-kube-api-access-rpt8s\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:02.646798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.646647    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:02.646798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.646661    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd6761d-5ede-4063-a9da-58c93f700d8b-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:02.646798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.646671    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e10f13-238f-4cf6-8602-8ed114eb6d1c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56e10f13-238f-4cf6-8602-8ed114eb6d1c" (UID: "56e10f13-238f-4cf6-8602-8ed114eb6d1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:17:02.648208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.648182    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56e10f13-238f-4cf6-8602-8ed114eb6d1c-kube-api-access-mxhc6" (OuterVolumeSpecName: "kube-api-access-mxhc6") pod "56e10f13-238f-4cf6-8602-8ed114eb6d1c" (UID: "56e10f13-238f-4cf6-8602-8ed114eb6d1c"). InnerVolumeSpecName "kube-api-access-mxhc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:02.648572 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.648554    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39f4400b-1923-44d5-b955-caf2c0b6b445-kube-api-access-nf45b" (OuterVolumeSpecName: "kube-api-access-nf45b") pod "39f4400b-1923-44d5-b955-caf2c0b6b445" (UID: "39f4400b-1923-44d5-b955-caf2c0b6b445"). InnerVolumeSpecName "kube-api-access-nf45b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:02.747968 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.747924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-certs\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.748145 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.747983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g2vv\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-kube-api-access-5g2vv\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.748145 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istiod-ca-cert\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.748145 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-token\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.748145 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-socket\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.748310 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-podinfo\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.748310 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-data\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.748310 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-credential-socket\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.748421 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-envoy\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.748456 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748420    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56e10f13-238f-4cf6-8602-8ed114eb6d1c-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:02.748456 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748433    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxhc6\" (UniqueName: \"kubernetes.io/projected/56e10f13-238f-4cf6-8602-8ed114eb6d1c-kube-api-access-mxhc6\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:02.748456 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748442    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf45b\" (UniqueName: \"kubernetes.io/projected/39f4400b-1923-44d5-b955-caf2c0b6b445-kube-api-access-nf45b\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:02.748456 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.748451    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f4400b-1923-44d5-b955-caf2c0b6b445-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:02.769684 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.769649    2563 generic.go:356] "Generic (PLEG): container finished" podID="edd6761d-5ede-4063-a9da-58c93f700d8b" containerID="5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03" exitCode=0
Sep 07 09:17:02.770430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.769710    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"
Sep 07 09:17:02.770430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.769739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl" event={"ID":"edd6761d-5ede-4063-a9da-58c93f700d8b","Type":"ContainerDied","Data":"5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03"}
Sep 07 09:17:02.770430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.769821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl" event={"ID":"edd6761d-5ede-4063-a9da-58c93f700d8b","Type":"ContainerDied","Data":"fb5d0b64cbbecbc2816d7ac4855641e5d93df4d30b167c19217ed27c1b14617a"}
Sep 07 09:17:02.770430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.769858    2563 scope.go:117] "RemoveContainer" containerID="5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03"
Sep 07 09:17:02.771665 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.771636    2563 generic.go:356] "Generic (PLEG): container finished" podID="39f4400b-1923-44d5-b955-caf2c0b6b445" containerID="b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584" exitCode=0
Sep 07 09:17:02.771766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.771688    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"
Sep 07 09:17:02.771766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.771735    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9" event={"ID":"39f4400b-1923-44d5-b955-caf2c0b6b445","Type":"ContainerDied","Data":"b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584"}
Sep 07 09:17:02.771892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.771798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9" event={"ID":"39f4400b-1923-44d5-b955-caf2c0b6b445","Type":"ContainerDied","Data":"631f2b93fe4a0c4a20520319b65a641f3501579919c3274f7f3dc586a60700d2"}
Sep 07 09:17:02.773387 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.773361    2563 generic.go:356] "Generic (PLEG): container finished" podID="56e10f13-238f-4cf6-8602-8ed114eb6d1c" containerID="ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a" exitCode=0
Sep 07 09:17:02.773510 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.773421    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk" event={"ID":"56e10f13-238f-4cf6-8602-8ed114eb6d1c","Type":"ContainerDied","Data":"ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a"}
Sep 07 09:17:02.773510 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.773423    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"
Sep 07 09:17:02.773510 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.773460    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk" event={"ID":"56e10f13-238f-4cf6-8602-8ed114eb6d1c","Type":"ContainerDied","Data":"b51c3241dc1155acfaadde410661922f9902ea3d5115157c0244d406e1e31d3f"}
Sep 07 09:17:02.782753 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.782395    2563 scope.go:117] "RemoveContainer" containerID="5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03"
Sep 07 09:17:02.782753 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:17:02.782717    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03\": container with ID starting with 5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03 not found: ID does not exist" containerID="5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03"
Sep 07 09:17:02.783090 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.782751    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03"} err="failed to get container status \"5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03\": rpc error: code = NotFound desc = could not find container \"5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03\": container with ID starting with 5b6745f84449be58df4b566ac0c416e594bd36aeb4777d6aedb606ed3652cd03 not found: ID does not exist"
Sep 07 09:17:02.783090 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.782775    2563 scope.go:117] "RemoveContainer" containerID="b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584"
Sep 07 09:17:02.792052 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.792034    2563 scope.go:117] "RemoveContainer" containerID="b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584"
Sep 07 09:17:02.792363 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:17:02.792344    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584\": container with ID starting with b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584 not found: ID does not exist" containerID="b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584"
Sep 07 09:17:02.792419 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.792370    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584"} err="failed to get container status \"b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584\": rpc error: code = NotFound desc = could not find container \"b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584\": container with ID starting with b0f72e0536feeea67f68d873ee63e28c6d4871bcabbffd268be366e314665584 not found: ID does not exist"
Sep 07 09:17:02.792419 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.792387    2563 scope.go:117] "RemoveContainer" containerID="ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a"
Sep 07 09:17:02.800407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.800383    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"]
Sep 07 09:17:02.801429 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.801407    2563 scope.go:117] "RemoveContainer" containerID="ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a"
Sep 07 09:17:02.801722 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:17:02.801703    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a\": container with ID starting with ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a not found: ID does not exist" containerID="ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a"
Sep 07 09:17:02.801777 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.801731    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a"} err="failed to get container status \"ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a\": rpc error: code = NotFound desc = could not find container \"ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a\": container with ID starting with ea26a6c728ab2cf545c7af968cdc0eeedf3eec0f60ecf5de182df4ec5615db7a not found: ID does not exist"
Sep 07 09:17:02.804825 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.804803    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-085b77-5f8d7bb75c-2jfl9"]
Sep 07 09:17:02.816551 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.816525    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"]
Sep 07 09:17:02.818449 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.818427    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-085b77-585cc5cf97-78cdl"]
Sep 07 09:17:02.827710 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.827691    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"]
Sep 07 09:17:02.831460 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.831432    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-085b77-58bd95fff4-6vkwk"]
Sep 07 09:17:02.849217 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-envoy\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-certs\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g2vv\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-kube-api-access-5g2vv\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istiod-ca-cert\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-token\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849389    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-socket\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849432    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-podinfo\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849619 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-data\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849619 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-credential-socket\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849740 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-certs\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.849963 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-data\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.850063 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.849946    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-socket\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.850063 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.850001    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-credential-socket\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.850063 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.850012    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istiod-ca-cert\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.851600 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.851572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-envoy\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.851787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.851769    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-podinfo\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.856760 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.856724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g2vv\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-kube-api-access-5g2vv\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.856850 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.856743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-token\") pod \"e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") " pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:02.901952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:02.901844    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:03.180101 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:03.180075    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"]
Sep 07 09:17:03.181914 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:17:03.181885    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1480e2be_60ab_4dc4_9f4e_bcb08de5539a.slice/crio-dae81097652b10b000c62a787cbc500405d543de39164e7c52d88fe70c432142 WatchSource:0}: Error finding container dae81097652b10b000c62a787cbc500405d543de39164e7c52d88fe70c432142: Status 404 returned error can't find the container with id dae81097652b10b000c62a787cbc500405d543de39164e7c52d88fe70c432142
Sep 07 09:17:03.184275 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:03.184232    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:17:03.184389 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:03.184366    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:17:03.184434 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:03.184421    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:17:03.781733 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:03.781684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" event={"ID":"1480e2be-60ab-4dc4-9f4e-bcb08de5539a","Type":"ContainerStarted","Data":"18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1"}
Sep 07 09:17:03.781924 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:03.781784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" event={"ID":"1480e2be-60ab-4dc4-9f4e-bcb08de5539a","Type":"ContainerStarted","Data":"dae81097652b10b000c62a787cbc500405d543de39164e7c52d88fe70c432142"}
Sep 07 09:17:03.800392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:03.800339    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" podStartSLOduration=1.800324112 podStartE2EDuration="1.800324112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:17:03.799071129 +0000 UTC m=+1081.768241770" watchObservedRunningTime="2026-09-07 09:17:03.800324112 +0000 UTC m=+1081.769494760"
Sep 07 09:17:03.902767 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:03.902724    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:04.465149 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:04.465112    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39f4400b-1923-44d5-b955-caf2c0b6b445" path="/var/lib/kubelet/pods/39f4400b-1923-44d5-b955-caf2c0b6b445/volumes"
Sep 07 09:17:04.465540 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:04.465525    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56e10f13-238f-4cf6-8602-8ed114eb6d1c" path="/var/lib/kubelet/pods/56e10f13-238f-4cf6-8602-8ed114eb6d1c/volumes"
Sep 07 09:17:04.465872 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:04.465860    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edd6761d-5ede-4063-a9da-58c93f700d8b" path="/var/lib/kubelet/pods/edd6761d-5ede-4063-a9da-58c93f700d8b/volumes"
Sep 07 09:17:04.908035 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:04.908004    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:05.791925 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:05.791894    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:05.792909 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:05.792889    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:06.746636 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.746596    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"]
Sep 07 09:17:06.751788 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.751760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.754370 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.754346    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e8e5dd53-serving-cert\""
Sep 07 09:17:06.759263 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.759238    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"]
Sep 07 09:17:06.887667 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.887617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzt55\" (UniqueName: \"kubernetes.io/projected/63d187da-8d73-478b-86fa-3828a4d675c3-kube-api-access-hzt55\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.888166 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.887768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-maas-api-tls\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.888166 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.887887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d187da-8d73-478b-86fa-3828a4d675c3-openshift-service-ca\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.888166 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.887920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-metrics-certs\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.988701 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.988659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzt55\" (UniqueName: \"kubernetes.io/projected/63d187da-8d73-478b-86fa-3828a4d675c3-kube-api-access-hzt55\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.988884 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.988707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-maas-api-tls\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.988884 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.988762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d187da-8d73-478b-86fa-3828a4d675c3-openshift-service-ca\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.988884 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.988793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-metrics-certs\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.988884 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:17:06.988865    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-serving-cert: secret "maas-api-e2e-mt-e8e5dd53-serving-cert" not found
Sep 07 09:17:06.989126 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:17:06.988933    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-maas-api-tls podName:63d187da-8d73-478b-86fa-3828a4d675c3 nodeName:}" failed. No retries permitted until 2026-09-07 09:17:07.488911394 +0000 UTC m=+1085.458081980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-maas-api-tls") pod "maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj" (UID: "63d187da-8d73-478b-86fa-3828a4d675c3") : secret "maas-api-e2e-mt-e8e5dd53-serving-cert" not found
Sep 07 09:17:06.989507 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.989482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d187da-8d73-478b-86fa-3828a4d675c3-openshift-service-ca\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:06.991329 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:06.991307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-metrics-certs\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:07.002837 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:07.002761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzt55\" (UniqueName: \"kubernetes.io/projected/63d187da-8d73-478b-86fa-3828a4d675c3-kube-api-access-hzt55\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:07.495752 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:07.495699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-maas-api-tls\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:07.499231 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:07.499199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-maas-api-tls\") pod \"maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:07.665166 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:07.665126    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:07.940378 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:07.940350    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"]
Sep 07 09:17:07.942182 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:17:07.942152    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d187da_8d73_478b_86fa_3828a4d675c3.slice/crio-964c90c1c007a645f38e28510b9cc2c7d611a477bca55191c1fa2d3491ab9cc9 WatchSource:0}: Error finding container 964c90c1c007a645f38e28510b9cc2c7d611a477bca55191c1fa2d3491ab9cc9: Status 404 returned error can't find the container with id 964c90c1c007a645f38e28510b9cc2c7d611a477bca55191c1fa2d3491ab9cc9
Sep 07 09:17:08.806985 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:08.806943    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj" event={"ID":"63d187da-8d73-478b-86fa-3828a4d675c3","Type":"ContainerStarted","Data":"0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b"}
Sep 07 09:17:08.807184 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:08.807002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj" event={"ID":"63d187da-8d73-478b-86fa-3828a4d675c3","Type":"ContainerStarted","Data":"964c90c1c007a645f38e28510b9cc2c7d611a477bca55191c1fa2d3491ab9cc9"}
Sep 07 09:17:08.807184 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:08.807146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:08.824407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:08.824358    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj" podStartSLOduration=2.410062741 podStartE2EDuration="2.82434241s" totalImagesPullingTime="414.279669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:17:08.822508609 +0000 UTC m=+1086.791679214" watchObservedRunningTime="2026-09-07 09:17:08.82434241 +0000 UTC m=+1086.793513014"
Sep 07 09:17:14.025005 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.024964    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"]
Sep 07 09:17:14.025608 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.025340    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy" containerID="cri-o://5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0" gracePeriod=30
Sep 07 09:17:14.745500 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.745463    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"]
Sep 07 09:17:14.750257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.750226    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.754606 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.754574    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-b0d2d0-openshift-default-dockercfg-mnm2f\""
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.770241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"]
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.777653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-envoy\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.777705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-credential-socket\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.777748    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-podinfo\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.777806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhgvl\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-kube-api-access-dhgvl\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.777959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-certs\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.778037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-token\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.778144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-data\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.778248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istiod-ca-cert\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.778332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.778338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-socket\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.816714 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.816686    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.880809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istiod-ca-cert\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.880872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-socket\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.880952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-envoy\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.880979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-credential-socket\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.881018    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-podinfo\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.881077    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhgvl\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-kube-api-access-dhgvl\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.881155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-certs\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.881198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-token\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.881230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-data\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.881530    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istiod-ca-cert\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.881655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-data\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.881775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-certs\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.881969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-socket\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.882680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.882219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-credential-socket\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.884047 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.884016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-podinfo\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.884499 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.884470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-envoy\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.890779 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.890744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-token\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:14.891468 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:14.891446    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhgvl\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-kube-api-access-dhgvl\") pod \"e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") " pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:15.085349 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:15.085229    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:15.376851 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:15.376714    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"]
Sep 07 09:17:15.379324 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:17:15.379277    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6596d6ff_c8c0_4a3f_8044_9928c6b48f9e.slice/crio-055556a2ed0059d03f93e5219ac184fe33213807427a759d0e6a479d1c695eca WatchSource:0}: Error finding container 055556a2ed0059d03f93e5219ac184fe33213807427a759d0e6a479d1c695eca: Status 404 returned error can't find the container with id 055556a2ed0059d03f93e5219ac184fe33213807427a759d0e6a479d1c695eca
Sep 07 09:17:15.382091 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:15.382054    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:17:15.382161 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:15.382123    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:17:15.382161 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:15.382155    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:17:15.843256 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:15.843207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt" event={"ID":"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e","Type":"ContainerStarted","Data":"3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647"}
Sep 07 09:17:15.843501 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:15.843298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt" event={"ID":"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e","Type":"ContainerStarted","Data":"055556a2ed0059d03f93e5219ac184fe33213807427a759d0e6a479d1c695eca"}
Sep 07 09:17:15.866194 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:15.866113    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt" podStartSLOduration=1.866089118 podStartE2EDuration="1.866089118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:17:15.862725316 +0000 UTC m=+1093.831895922" watchObservedRunningTime="2026-09-07 09:17:15.866089118 +0000 UTC m=+1093.835259724"
Sep 07 09:17:16.085552 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:16.085516    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:16.090943 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:16.090917    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:16.851698 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:16.851670    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:16.856727 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:16.856703    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:19.318931 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.318902    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:17:19.427896 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.427854    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-envoy\") pod \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") "
Sep 07 09:17:19.427896 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.427903    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-token\") pod \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") "
Sep 07 09:17:19.428151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.427942    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istiod-ca-cert\") pod \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") "
Sep 07 09:17:19.428151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.427981    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-credential-socket\") pod \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") "
Sep 07 09:17:19.428151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428000    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-socket\") pod \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") "
Sep 07 09:17:19.428151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428019    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-podinfo\") pod \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") "
Sep 07 09:17:19.428151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428057    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-data\") pod \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") "
Sep 07 09:17:19.428151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428104    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-728bg\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-kube-api-access-728bg\") pod \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") "
Sep 07 09:17:19.428151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428143    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-certs\") pod \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\" (UID: \"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc\") "
Sep 07 09:17:19.428561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428278    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" (UID: "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:19.428561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428326    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" (UID: "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:19.428561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428432    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-data" (OuterVolumeSpecName: "istio-data") pod "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" (UID: "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:19.428561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428495    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" (UID: "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:19.428561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428520    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" (UID: "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:17:19.428561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428540    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:19.428561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428563    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:19.428922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.428577    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:19.430104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.430080    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" (UID: "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:19.430224 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.430208    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-token" (OuterVolumeSpecName: "istio-token") pod "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" (UID: "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:19.430319 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.430268    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-kube-api-access-728bg" (OuterVolumeSpecName: "kube-api-access-728bg") pod "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" (UID: "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc"). InnerVolumeSpecName "kube-api-access-728bg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:19.430402 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.430387    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" (UID: "d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:17:19.529903 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.529865    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:19.529903 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.529896    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:19.529903 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.529905    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:19.529903 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.529913    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:19.530177 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.529920    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:19.530177 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.529929    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-728bg\" (UniqueName: \"kubernetes.io/projected/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc-kube-api-access-728bg\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:19.874771 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.874733    2563 generic.go:356] "Generic (PLEG): container finished" podID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerID="5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0" exitCode=0
Sep 07 09:17:19.874952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.874780    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"
Sep 07 09:17:19.874952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.874810    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" event={"ID":"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc","Type":"ContainerDied","Data":"5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0"}
Sep 07 09:17:19.874952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.874866    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq" event={"ID":"d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc","Type":"ContainerDied","Data":"be4d39ebe78b3b6361072638fca2f99270608eb89f0b1d0158f856276e3f95f7"}
Sep 07 09:17:19.874952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.874888    2563 scope.go:117] "RemoveContainer" containerID="5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0"
Sep 07 09:17:19.884656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.884640    2563 scope.go:117] "RemoveContainer" containerID="e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da"
Sep 07 09:17:19.893680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.893655    2563 scope.go:117] "RemoveContainer" containerID="5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0"
Sep 07 09:17:19.893949 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:17:19.893930    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0\": container with ID starting with 5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0 not found: ID does not exist" containerID="5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0"
Sep 07 09:17:19.894000 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.893958    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0"} err="failed to get container status \"5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0\": rpc error: code = NotFound desc = could not find container \"5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0\": container with ID starting with 5ffc582a3c8acdd792c58b81724a29680ae26106afa8b2c0a4188227624b8cb0 not found: ID does not exist"
Sep 07 09:17:19.894000 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.893977    2563 scope.go:117] "RemoveContainer" containerID="e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da"
Sep 07 09:17:19.894234 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:17:19.894216    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da\": container with ID starting with e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da not found: ID does not exist" containerID="e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da"
Sep 07 09:17:19.894305 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.894241    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da"} err="failed to get container status \"e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da\": rpc error: code = NotFound desc = could not find container \"e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da\": container with ID starting with e14ed737eb6ee8b9dc9ff022b145d72bec76c56ce9a11793759212b34ac487da not found: ID does not exist"
Sep 07 09:17:19.910871 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.910842    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"]
Sep 07 09:17:19.923908 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:19.923881    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-085b77-openshift-default-74dbf4bc76-ncmdq"]
Sep 07 09:17:20.464782 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:20.464747    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" path="/var/lib/kubelet/pods/d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc/volumes"
Sep 07 09:17:21.833447 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:21.833407    2563 patch_prober.go:28] interesting pod/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 09:17:21.833826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:21.833476    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 09:17:23.896464 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:23.896426    2563 generic.go:356] "Generic (PLEG): container finished" podID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerID="18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1" exitCode=0
Sep 07 09:17:23.896898 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:23.896479    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" event={"ID":"1480e2be-60ab-4dc4-9f4e-bcb08de5539a","Type":"ContainerDied","Data":"18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1"}
Sep 07 09:17:23.896942 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:23.896901    2563 scope.go:117] "RemoveContainer" containerID="18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1"
Sep 07 09:17:23.897989 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:23.897958    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:17:23.898094 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:23.898027    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:17:23.898094 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:23.898063    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:17:24.903070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:24.903030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" event={"ID":"1480e2be-60ab-4dc4-9f4e-bcb08de5539a","Type":"ContainerStarted","Data":"e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c"}
Sep 07 09:17:25.902974 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:25.902933    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:25.903166 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:25.903090    2563 patch_prober.go:28] interesting pod/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 07 09:17:25.903469 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:25.903154    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 07 09:17:26.902850 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:26.902816    2563 patch_prober.go:28] interesting pod/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 07 09:17:26.903026 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:26.902883    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 07 09:17:27.903140 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:27.903096    2563 patch_prober.go:28] interesting pod/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 07 09:17:27.903650 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:27.903159    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 07 09:17:28.902942 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:28.902900    2563 patch_prober.go:28] interesting pod/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 07 09:17:28.903130 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:28.902979    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 07 09:17:30.005426 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:30.005382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:30.006160 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:30.005879    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:30.007848 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:30.007823    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:17:40.025935 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:40.025903    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"]
Sep 07 09:17:40.026371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:40.026108    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt" podUID="6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" containerName="istio-proxy" containerID="cri-o://3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647" gracePeriod=30
Sep 07 09:17:45.266301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.266258    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:45.375130 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375033    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-podinfo\") pod \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") "
Sep 07 09:17:45.375130 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375091    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-certs\") pod \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") "
Sep 07 09:17:45.375130 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375119    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-socket\") pod \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") "
Sep 07 09:17:45.375467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375155    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-token\") pod \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") "
Sep 07 09:17:45.375467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375199    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-envoy\") pod \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") "
Sep 07 09:17:45.375467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375220    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-credential-socket\") pod \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") "
Sep 07 09:17:45.375467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375257    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhgvl\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-kube-api-access-dhgvl\") pod \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") "
Sep 07 09:17:45.375467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375323    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istiod-ca-cert\") pod \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") "
Sep 07 09:17:45.375467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375370    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-data\") pod \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\" (UID: \"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e\") "
Sep 07 09:17:45.375467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375442    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" (UID: "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:45.375819 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375488    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" (UID: "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:45.375819 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375711    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-data" (OuterVolumeSpecName: "istio-data") pod "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" (UID: "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:45.375946 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375869    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" (UID: "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:45.375946 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.375875    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" (UID: "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:17:45.376527 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.376503    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:45.376593 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.376532    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:45.376593 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.376548    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:45.376593 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.376562    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:45.376593 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.376575    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:45.377569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.377507    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" (UID: "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:17:45.377698 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.377561    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" (UID: "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:45.377785 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.377769    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" (UID: "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:45.377865 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.377850    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-kube-api-access-dhgvl" (OuterVolumeSpecName: "kube-api-access-dhgvl") pod "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" (UID: "6596d6ff-c8c0-4a3f-8044-9928c6b48f9e"). InnerVolumeSpecName "kube-api-access-dhgvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:45.477795 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.477750    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:45.477795 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.477786    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:45.478093 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.477827    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhgvl\" (UniqueName: \"kubernetes.io/projected/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-kube-api-access-dhgvl\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:45.478093 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:45.477838    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:46.011459 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.011420    2563 generic.go:356] "Generic (PLEG): container finished" podID="6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" containerID="3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647" exitCode=0
Sep 07 09:17:46.011643 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.011477    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"
Sep 07 09:17:46.011643 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.011499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt" event={"ID":"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e","Type":"ContainerDied","Data":"3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647"}
Sep 07 09:17:46.011643 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.011555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt" event={"ID":"6596d6ff-c8c0-4a3f-8044-9928c6b48f9e","Type":"ContainerDied","Data":"055556a2ed0059d03f93e5219ac184fe33213807427a759d0e6a479d1c695eca"}
Sep 07 09:17:46.011643 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.011584    2563 scope.go:117] "RemoveContainer" containerID="3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647"
Sep 07 09:17:46.020675 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.020658    2563 scope.go:117] "RemoveContainer" containerID="3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647"
Sep 07 09:17:46.020962 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:17:46.020941    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647\": container with ID starting with 3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647 not found: ID does not exist" containerID="3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647"
Sep 07 09:17:46.021005 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.020970    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647"} err="failed to get container status \"3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647\": rpc error: code = NotFound desc = could not find container \"3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647\": container with ID starting with 3c68c1db42f09333605698658d4aff8d9d38dbbabef3bfb2d50219612f764647 not found: ID does not exist"
Sep 07 09:17:46.033529 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.033499    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"]
Sep 07 09:17:46.037576 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.037548    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-b0d2d0-openshift-default-54cd7ff9d4-s79jt"]
Sep 07 09:17:46.465039 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:46.464954    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" path="/var/lib/kubelet/pods/6596d6ff-c8c0-4a3f-8044-9928c6b48f9e/volumes"
Sep 07 09:17:48.085397 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.085352    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"]
Sep 07 09:17:48.085834 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.085611    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj" podUID="63d187da-8d73-478b-86fa-3828a4d675c3" containerName="maas-api" containerID="cri-o://0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b" gracePeriod=30
Sep 07 09:17:48.343009 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.342936    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:48.504579 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.504547    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzt55\" (UniqueName: \"kubernetes.io/projected/63d187da-8d73-478b-86fa-3828a4d675c3-kube-api-access-hzt55\") pod \"63d187da-8d73-478b-86fa-3828a4d675c3\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") "
Sep 07 09:17:48.504772 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.504604    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-metrics-certs\") pod \"63d187da-8d73-478b-86fa-3828a4d675c3\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") "
Sep 07 09:17:48.504772 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.504695    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d187da-8d73-478b-86fa-3828a4d675c3-openshift-service-ca\") pod \"63d187da-8d73-478b-86fa-3828a4d675c3\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") "
Sep 07 09:17:48.504772 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.504720    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-maas-api-tls\") pod \"63d187da-8d73-478b-86fa-3828a4d675c3\" (UID: \"63d187da-8d73-478b-86fa-3828a4d675c3\") "
Sep 07 09:17:48.505088 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.505056    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63d187da-8d73-478b-86fa-3828a4d675c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63d187da-8d73-478b-86fa-3828a4d675c3" (UID: "63d187da-8d73-478b-86fa-3828a4d675c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:17:48.506774 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.506744    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "63d187da-8d73-478b-86fa-3828a4d675c3" (UID: "63d187da-8d73-478b-86fa-3828a4d675c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:17:48.506880 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.506792    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63d187da-8d73-478b-86fa-3828a4d675c3" (UID: "63d187da-8d73-478b-86fa-3828a4d675c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:17:48.506880 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.506817    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63d187da-8d73-478b-86fa-3828a4d675c3-kube-api-access-hzt55" (OuterVolumeSpecName: "kube-api-access-hzt55") pod "63d187da-8d73-478b-86fa-3828a4d675c3" (UID: "63d187da-8d73-478b-86fa-3828a4d675c3"). InnerVolumeSpecName "kube-api-access-hzt55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:48.606029 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.605956    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzt55\" (UniqueName: \"kubernetes.io/projected/63d187da-8d73-478b-86fa-3828a4d675c3-kube-api-access-hzt55\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:48.606029 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.605980    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:48.606029 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.605989    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d187da-8d73-478b-86fa-3828a4d675c3-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:48.606029 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:48.606000    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d187da-8d73-478b-86fa-3828a4d675c3-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:17:49.034508 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:49.034470    2563 generic.go:356] "Generic (PLEG): container finished" podID="63d187da-8d73-478b-86fa-3828a4d675c3" containerID="0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b" exitCode=0
Sep 07 09:17:49.034685 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:49.034536    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"
Sep 07 09:17:49.034685 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:49.034548    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj" event={"ID":"63d187da-8d73-478b-86fa-3828a4d675c3","Type":"ContainerDied","Data":"0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b"}
Sep 07 09:17:49.034685 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:49.034640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj" event={"ID":"63d187da-8d73-478b-86fa-3828a4d675c3","Type":"ContainerDied","Data":"964c90c1c007a645f38e28510b9cc2c7d611a477bca55191c1fa2d3491ab9cc9"}
Sep 07 09:17:49.034685 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:49.034670    2563 scope.go:117] "RemoveContainer" containerID="0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b"
Sep 07 09:17:49.044454 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:49.044428    2563 scope.go:117] "RemoveContainer" containerID="0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b"
Sep 07 09:17:49.044796 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:17:49.044774    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b\": container with ID starting with 0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b not found: ID does not exist" containerID="0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b"
Sep 07 09:17:49.044872 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:49.044803    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b"} err="failed to get container status \"0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b\": rpc error: code = NotFound desc = could not find container \"0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b\": container with ID starting with 0b327627991f4d6c9d047098b981c9808288e0939b5dbb044685f940f705821b not found: ID does not exist"
Sep 07 09:17:49.056851 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:49.056820    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"]
Sep 07 09:17:49.058448 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:49.058426    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e8e5dd53-54bc95fb88-nrnxj"]
Sep 07 09:17:50.465947 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:50.465910    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63d187da-8d73-478b-86fa-3828a4d675c3" path="/var/lib/kubelet/pods/63d187da-8d73-478b-86fa-3828a4d675c3/volumes"
Sep 07 09:17:54.763312 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:54.763262    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"]
Sep 07 09:17:54.763785 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:17:54.763516    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy" containerID="cri-o://e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c" gracePeriod=30
Sep 07 09:18:00.006310 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.006268    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:18:00.015049 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015025    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-podinfo\") pod \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") "
Sep 07 09:18:00.015170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-credential-socket\") pod \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") "
Sep 07 09:18:00.015170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015078    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g2vv\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-kube-api-access-5g2vv\") pod \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") "
Sep 07 09:18:00.015170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015107    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-token\") pod \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") "
Sep 07 09:18:00.015170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-certs\") pod \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") "
Sep 07 09:18:00.015170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015154    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istiod-ca-cert\") pod \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") "
Sep 07 09:18:00.015465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015177    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-socket\") pod \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") "
Sep 07 09:18:00.015465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015226    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-data\") pod \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") "
Sep 07 09:18:00.015465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015299    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-envoy\") pod \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\" (UID: \"1480e2be-60ab-4dc4-9f4e-bcb08de5539a\") "
Sep 07 09:18:00.015465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015376    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1480e2be-60ab-4dc4-9f4e-bcb08de5539a" (UID: "1480e2be-60ab-4dc4-9f4e-bcb08de5539a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:00.015465 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015450    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1480e2be-60ab-4dc4-9f4e-bcb08de5539a" (UID: "1480e2be-60ab-4dc4-9f4e-bcb08de5539a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:00.015790 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015606    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-data" (OuterVolumeSpecName: "istio-data") pod "1480e2be-60ab-4dc4-9f4e-bcb08de5539a" (UID: "1480e2be-60ab-4dc4-9f4e-bcb08de5539a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:00.015790 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015613    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:00.015790 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015631    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:00.015790 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015624    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1480e2be-60ab-4dc4-9f4e-bcb08de5539a" (UID: "1480e2be-60ab-4dc4-9f4e-bcb08de5539a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:00.015790 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.015683    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1480e2be-60ab-4dc4-9f4e-bcb08de5539a" (UID: "1480e2be-60ab-4dc4-9f4e-bcb08de5539a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:18:00.017392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.017359    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-kube-api-access-5g2vv" (OuterVolumeSpecName: "kube-api-access-5g2vv") pod "1480e2be-60ab-4dc4-9f4e-bcb08de5539a" (UID: "1480e2be-60ab-4dc4-9f4e-bcb08de5539a"). InnerVolumeSpecName "kube-api-access-5g2vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:00.017512 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.017413    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-token" (OuterVolumeSpecName: "istio-token") pod "1480e2be-60ab-4dc4-9f4e-bcb08de5539a" (UID: "1480e2be-60ab-4dc4-9f4e-bcb08de5539a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:00.017733 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.017711    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1480e2be-60ab-4dc4-9f4e-bcb08de5539a" (UID: "1480e2be-60ab-4dc4-9f4e-bcb08de5539a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:00.017733 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.017723    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1480e2be-60ab-4dc4-9f4e-bcb08de5539a" (UID: "1480e2be-60ab-4dc4-9f4e-bcb08de5539a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:18:00.092000 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.091911    2563 generic.go:356] "Generic (PLEG): container finished" podID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerID="e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c" exitCode=0
Sep 07 09:18:00.092000 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.091967    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"
Sep 07 09:18:00.092191 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.091995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" event={"ID":"1480e2be-60ab-4dc4-9f4e-bcb08de5539a","Type":"ContainerDied","Data":"e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c"}
Sep 07 09:18:00.092191 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.092054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn" event={"ID":"1480e2be-60ab-4dc4-9f4e-bcb08de5539a","Type":"ContainerDied","Data":"dae81097652b10b000c62a787cbc500405d543de39164e7c52d88fe70c432142"}
Sep 07 09:18:00.092191 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.092077    2563 scope.go:117] "RemoveContainer" containerID="e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c"
Sep 07 09:18:00.101049 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.101033    2563 scope.go:117] "RemoveContainer" containerID="18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1"
Sep 07 09:18:00.111530 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.111513    2563 scope.go:117] "RemoveContainer" containerID="e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c"
Sep 07 09:18:00.111835 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:18:00.111816    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c\": container with ID starting with e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c not found: ID does not exist" containerID="e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c"
Sep 07 09:18:00.111890 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.111843    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c"} err="failed to get container status \"e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c\": rpc error: code = NotFound desc = could not find container \"e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c\": container with ID starting with e5cc707b90f4d6b04149653202253259775ac89b7aebac123cadf1b8b339f27c not found: ID does not exist"
Sep 07 09:18:00.111890 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.111871    2563 scope.go:117] "RemoveContainer" containerID="18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1"
Sep 07 09:18:00.112122 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:18:00.112105    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1\": container with ID starting with 18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1 not found: ID does not exist" containerID="18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1"
Sep 07 09:18:00.112196 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.112125    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1"} err="failed to get container status \"18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1\": rpc error: code = NotFound desc = could not find container \"18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1\": container with ID starting with 18b740e2892097c51fb73f68c4040fb3230c308baa8822ba22cd1b61872956e1 not found: ID does not exist"
Sep 07 09:18:00.114544 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.114523    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"]
Sep 07 09:18:00.116378 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.116352    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:00.116464 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.116382    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g2vv\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-kube-api-access-5g2vv\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:00.116464 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.116399    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:00.116464 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.116413    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:00.116464 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.116426    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:00.116464 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.116439    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:00.116464 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.116451    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1480e2be-60ab-4dc4-9f4e-bcb08de5539a-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:00.118026 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.118007    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e8e5dd53-openshift-default-59f66cb69b-bkggn"]
Sep 07 09:18:00.464889 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:00.464806    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" path="/var/lib/kubelet/pods/1480e2be-60ab-4dc4-9f4e-bcb08de5539a/volumes"
Sep 07 09:18:01.291780 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.291743    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"]
Sep 07 09:18:01.292249 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.291993    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92" podUID="0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" containerName="maas-api" containerID="cri-o://a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290" gracePeriod=30
Sep 07 09:18:01.347733 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.347689    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"]
Sep 07 09:18:01.348060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.348027    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f" podUID="7fa1d525-101d-41af-9391-8438ee1d0fe0" containerName="payload-processing" containerID="cri-o://242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a" gracePeriod=30
Sep 07 09:18:01.625703 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.625668    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:18:01.629231 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.629208    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-metrics-certs\") pod \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") "
Sep 07 09:18:01.629391 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.629261    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-openshift-service-ca\") pod \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") "
Sep 07 09:18:01.629391 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.629343    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gthdl\" (UniqueName: \"kubernetes.io/projected/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-kube-api-access-gthdl\") pod \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") "
Sep 07 09:18:01.629391 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.629377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-maas-api-tls\") pod \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\" (UID: \"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e\") "
Sep 07 09:18:01.629560 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.629428    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:18:01.629744 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.629710    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" (UID: "0cbe9963-8eb1-4f8b-8de3-b8b10013f42e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:18:01.631497 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.631469    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-kube-api-access-gthdl" (OuterVolumeSpecName: "kube-api-access-gthdl") pod "0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" (UID: "0cbe9963-8eb1-4f8b-8de3-b8b10013f42e"). InnerVolumeSpecName "kube-api-access-gthdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:01.631614 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.631529    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" (UID: "0cbe9963-8eb1-4f8b-8de3-b8b10013f42e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:18:01.631614 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.631540    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" (UID: "0cbe9963-8eb1-4f8b-8de3-b8b10013f42e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:18:01.730197 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.730156    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fa1d525-101d-41af-9391-8438ee1d0fe0-plugins-config-volume\") pod \"7fa1d525-101d-41af-9391-8438ee1d0fe0\" (UID: \"7fa1d525-101d-41af-9391-8438ee1d0fe0\") "
Sep 07 09:18:01.730197 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.730200    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw2ts\" (UniqueName: \"kubernetes.io/projected/7fa1d525-101d-41af-9391-8438ee1d0fe0-kube-api-access-xw2ts\") pod \"7fa1d525-101d-41af-9391-8438ee1d0fe0\" (UID: \"7fa1d525-101d-41af-9391-8438ee1d0fe0\") "
Sep 07 09:18:01.730500 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.730381    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:01.730500 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.730393    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gthdl\" (UniqueName: \"kubernetes.io/projected/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-kube-api-access-gthdl\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:01.730500 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.730403    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:01.730500 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.730412    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:01.730685 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.730598    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fa1d525-101d-41af-9391-8438ee1d0fe0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7fa1d525-101d-41af-9391-8438ee1d0fe0" (UID: "7fa1d525-101d-41af-9391-8438ee1d0fe0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:18:01.732255 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.732233    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fa1d525-101d-41af-9391-8438ee1d0fe0-kube-api-access-xw2ts" (OuterVolumeSpecName: "kube-api-access-xw2ts") pod "7fa1d525-101d-41af-9391-8438ee1d0fe0" (UID: "7fa1d525-101d-41af-9391-8438ee1d0fe0"). InnerVolumeSpecName "kube-api-access-xw2ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:01.831231 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.831146    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fa1d525-101d-41af-9391-8438ee1d0fe0-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:01.831231 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:01.831177    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw2ts\" (UniqueName: \"kubernetes.io/projected/7fa1d525-101d-41af-9391-8438ee1d0fe0-kube-api-access-xw2ts\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:02.105250 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.105153    2563 generic.go:356] "Generic (PLEG): container finished" podID="7fa1d525-101d-41af-9391-8438ee1d0fe0" containerID="242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a" exitCode=0
Sep 07 09:18:02.105250 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.105199    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"
Sep 07 09:18:02.105493 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.105239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f" event={"ID":"7fa1d525-101d-41af-9391-8438ee1d0fe0","Type":"ContainerDied","Data":"242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a"}
Sep 07 09:18:02.105493 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.105320    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f" event={"ID":"7fa1d525-101d-41af-9391-8438ee1d0fe0","Type":"ContainerDied","Data":"d638c41fd0189c54f74e120f1432f40f14e11fd44ae925a48a1ccfc1a09a0cc4"}
Sep 07 09:18:02.105493 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.105344    2563 scope.go:117] "RemoveContainer" containerID="242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a"
Sep 07 09:18:02.106840 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.106810    2563 generic.go:356] "Generic (PLEG): container finished" podID="0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" containerID="a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290" exitCode=0
Sep 07 09:18:02.106958 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.106863    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"
Sep 07 09:18:02.106958 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.106889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92" event={"ID":"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e","Type":"ContainerDied","Data":"a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290"}
Sep 07 09:18:02.106958 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.106945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92" event={"ID":"0cbe9963-8eb1-4f8b-8de3-b8b10013f42e","Type":"ContainerDied","Data":"550b23fe5705c9c23b821de2a66785ac633c2ea148d6d34d40f75b67d5256f7d"}
Sep 07 09:18:02.114391 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.114344    2563 scope.go:117] "RemoveContainer" containerID="242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a"
Sep 07 09:18:02.114639 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:18:02.114619    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a\": container with ID starting with 242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a not found: ID does not exist" containerID="242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a"
Sep 07 09:18:02.114720 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.114648    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a"} err="failed to get container status \"242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a\": rpc error: code = NotFound desc = could not find container \"242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a\": container with ID starting with 242c18cc853b43eb86e1ac300d32937848119261a6869a56879d123f08076d6a not found: ID does not exist"
Sep 07 09:18:02.114720 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.114665    2563 scope.go:117] "RemoveContainer" containerID="a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290"
Sep 07 09:18:02.123097 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.123080    2563 scope.go:117] "RemoveContainer" containerID="a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290"
Sep 07 09:18:02.123395 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:18:02.123377    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290\": container with ID starting with a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290 not found: ID does not exist" containerID="a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290"
Sep 07 09:18:02.123453 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.123402    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290"} err="failed to get container status \"a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290\": rpc error: code = NotFound desc = could not find container \"a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290\": container with ID starting with a0bf50a82e7805c64682c12a41a42da46fece4da0bde90071d3de125d58b9290 not found: ID does not exist"
Sep 07 09:18:02.132069 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.132045    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"]
Sep 07 09:18:02.136096 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.136072    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-614a02-7b449b79f-n7j92"]
Sep 07 09:18:02.145802 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.145772    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"]
Sep 07 09:18:02.149810 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.149787    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-614a02-6d4f5678b9-42c2f"]
Sep 07 09:18:02.465009 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.464919    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" path="/var/lib/kubelet/pods/0cbe9963-8eb1-4f8b-8de3-b8b10013f42e/volumes"
Sep 07 09:18:02.465505 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:02.465482    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fa1d525-101d-41af-9391-8438ee1d0fe0" path="/var/lib/kubelet/pods/7fa1d525-101d-41af-9391-8438ee1d0fe0/volumes"
Sep 07 09:18:08.238729 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.238691    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"]
Sep 07 09:18:08.239202 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.238929    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy" containerID="cri-o://085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba" gracePeriod=30
Sep 07 09:18:08.701240 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701206    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"]
Sep 07 09:18:08.701591 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701578    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy"
Sep 07 09:18:08.701641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701593    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy"
Sep 07 09:18:08.701641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701621    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy"
Sep 07 09:18:08.701641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701628    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy"
Sep 07 09:18:08.701641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701635    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63d187da-8d73-478b-86fa-3828a4d675c3" containerName="maas-api"
Sep 07 09:18:08.701641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701640    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="63d187da-8d73-478b-86fa-3828a4d675c3" containerName="maas-api"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701649    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fa1d525-101d-41af-9391-8438ee1d0fe0" containerName="payload-processing"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701655    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fa1d525-101d-41af-9391-8438ee1d0fe0" containerName="payload-processing"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701665    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" containerName="maas-api"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701670    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" containerName="maas-api"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701679    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701684    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701692    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" containerName="istio-proxy"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701698    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" containerName="istio-proxy"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701707    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701713    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701765    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fa1d525-101d-41af-9391-8438ee1d0fe0" containerName="payload-processing"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701773    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cbe9963-8eb1-4f8b-8de3-b8b10013f42e" containerName="maas-api"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701778    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701784    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6596d6ff-c8c0-4a3f-8044-9928c6b48f9e" containerName="istio-proxy"
Sep 07 09:18:08.701787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701791    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy"
Sep 07 09:18:08.702218 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701796    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1480e2be-60ab-4dc4-9f4e-bcb08de5539a" containerName="istio-proxy"
Sep 07 09:18:08.702218 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.701802    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="63d187da-8d73-478b-86fa-3828a4d675c3" containerName="maas-api"
Sep 07 09:18:08.703729 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.703712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.706346 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.706323    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-263421-openshift-default-dockercfg-6tdr6\""
Sep 07 09:18:08.718577 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.718552    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"]
Sep 07 09:18:08.798490 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.798454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-data\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.798490 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.798494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-socket\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.798735 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.798536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-envoy\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.798735 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.798585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-certs\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.798735 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.798603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-token\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.798735 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.798620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-podinfo\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.798735 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.798711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-credential-socket\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.798963 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.798766    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308afb0b-413a-46ce-bbd4-9ef6d000049d-istiod-ca-cert\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.798963 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.798826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f46pp\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-kube-api-access-f46pp\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900024 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.899974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308afb0b-413a-46ce-bbd4-9ef6d000049d-istiod-ca-cert\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46pp\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-kube-api-access-f46pp\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-data\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-socket\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900124    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-envoy\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-certs\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-token\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-podinfo\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900645 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-credential-socket\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900645 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900615    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-certs\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900751 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900685    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-socket\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900751 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-credential-socket\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900825 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-data\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.900825 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.900795    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308afb0b-413a-46ce-bbd4-9ef6d000049d-istiod-ca-cert\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.902630 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.902598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-envoy\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.902899 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.902879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-podinfo\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.908366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.908342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f46pp\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-kube-api-access-f46pp\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:08.908541 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:08.908525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-token\") pod \"e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") " pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:09.014028 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:09.013994    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:09.305673 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:09.305647    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"]
Sep 07 09:18:09.308140 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:18:09.308102    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308afb0b_413a_46ce_bbd4_9ef6d000049d.slice/crio-bcf08b198af3b8ff1f0b1cfb45f110366f069636dfbe1d50dc4054356b099ef9 WatchSource:0}: Error finding container bcf08b198af3b8ff1f0b1cfb45f110366f069636dfbe1d50dc4054356b099ef9: Status 404 returned error can't find the container with id bcf08b198af3b8ff1f0b1cfb45f110366f069636dfbe1d50dc4054356b099ef9
Sep 07 09:18:09.310852 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:09.310808    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:09.310926 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:09.310903    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:09.310970 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:09.310949    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:10.150132 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:10.150090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" event={"ID":"308afb0b-413a-46ce-bbd4-9ef6d000049d","Type":"ContainerStarted","Data":"8559e36e712cbc561a6aead535392755ff300984770025edff4e71900041f723"}
Sep 07 09:18:10.150322 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:10.150159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" event={"ID":"308afb0b-413a-46ce-bbd4-9ef6d000049d","Type":"ContainerStarted","Data":"bcf08b198af3b8ff1f0b1cfb45f110366f069636dfbe1d50dc4054356b099ef9"}
Sep 07 09:18:10.172483 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:10.172427    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" podStartSLOduration=2.172411394 podStartE2EDuration="2.172411394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:10.171755976 +0000 UTC m=+1148.140926597" watchObservedRunningTime="2026-09-07 09:18:10.172411394 +0000 UTC m=+1148.141581999"
Sep 07 09:18:11.014203 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:11.014161    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:11.019398 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:11.019366    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:11.155139 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:11.155106    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:11.156211 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:11.156186    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:13.305046 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.305008    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"]
Sep 07 09:18:13.305540 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.305468    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d273cb58-8cb9-46fa-9e9c-5fd81f0a3ddc" containerName="istio-proxy"
Sep 07 09:18:13.307677 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.307655    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.310370 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.310344    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-263421-serving-cert\""
Sep 07 09:18:13.321789 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.321766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"]
Sep 07 09:18:13.340436 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.340401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbf77\" (UniqueName: \"kubernetes.io/projected/5de4ac61-5425-4f69-ab7a-ca8e319579eb-kube-api-access-jbf77\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.340724 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.340697    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.340922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.340896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.341053 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.340951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de4ac61-5425-4f69-ab7a-ca8e319579eb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.441896 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.441863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de4ac61-5425-4f69-ab7a-ca8e319579eb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.442087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.441911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbf77\" (UniqueName: \"kubernetes.io/projected/5de4ac61-5425-4f69-ab7a-ca8e319579eb-kube-api-access-jbf77\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.442087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.441996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.442087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.442021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.442251 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:18:13.442117    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-serving-cert: secret "maas-api-e2e-shared-a-w2-263421-serving-cert" not found
Sep 07 09:18:13.442251 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:18:13.442186    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-maas-api-tls podName:5de4ac61-5425-4f69-ab7a-ca8e319579eb nodeName:}" failed. No retries permitted until 2026-09-07 09:18:13.942165523 +0000 UTC m=+1151.911336110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-maas-api-tls") pod "maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6" (UID: "5de4ac61-5425-4f69-ab7a-ca8e319579eb") : secret "maas-api-e2e-shared-a-w2-263421-serving-cert" not found
Sep 07 09:18:13.442615 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.442592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de4ac61-5425-4f69-ab7a-ca8e319579eb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.444436 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.444416    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.450641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.450619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbf77\" (UniqueName: \"kubernetes.io/projected/5de4ac61-5425-4f69-ab7a-ca8e319579eb-kube-api-access-jbf77\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.490549 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.490521    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:18:13.542961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.542863    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ztrv\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-kube-api-access-6ztrv\") pod \"26a709fe-0870-4529-905c-5075b7758948\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") "
Sep 07 09:18:13.542961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.542935    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-credential-socket\") pod \"26a709fe-0870-4529-905c-5075b7758948\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") "
Sep 07 09:18:13.543194 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.542966    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a709fe-0870-4529-905c-5075b7758948-istio-podinfo\") pod \"26a709fe-0870-4529-905c-5075b7758948\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") "
Sep 07 09:18:13.543194 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.542987    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-socket\") pod \"26a709fe-0870-4529-905c-5075b7758948\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") "
Sep 07 09:18:13.543194 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-istio-token\") pod \"26a709fe-0870-4529-905c-5075b7758948\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") "
Sep 07 09:18:13.543194 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543089    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-certs\") pod \"26a709fe-0870-4529-905c-5075b7758948\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") "
Sep 07 09:18:13.543194 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a709fe-0870-4529-905c-5075b7758948-istiod-ca-cert\") pod \"26a709fe-0870-4529-905c-5075b7758948\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") "
Sep 07 09:18:13.543194 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543165    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-data\") pod \"26a709fe-0870-4529-905c-5075b7758948\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") "
Sep 07 09:18:13.543194 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543193    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-envoy\") pod \"26a709fe-0870-4529-905c-5075b7758948\" (UID: \"26a709fe-0870-4529-905c-5075b7758948\") "
Sep 07 09:18:13.543576 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543322    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "26a709fe-0870-4529-905c-5075b7758948" (UID: "26a709fe-0870-4529-905c-5075b7758948"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:13.543576 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543440    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "26a709fe-0870-4529-905c-5075b7758948" (UID: "26a709fe-0870-4529-905c-5075b7758948"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:13.543680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543662    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "26a709fe-0870-4529-905c-5075b7758948" (UID: "26a709fe-0870-4529-905c-5075b7758948"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:13.543743 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543687    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26a709fe-0870-4529-905c-5075b7758948-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "26a709fe-0870-4529-905c-5075b7758948" (UID: "26a709fe-0870-4529-905c-5075b7758948"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:18:13.543969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543871    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-data" (OuterVolumeSpecName: "istio-data") pod "26a709fe-0870-4529-905c-5075b7758948" (UID: "26a709fe-0870-4529-905c-5075b7758948"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:13.543969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543919    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:13.543969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.543944    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:13.545276 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.545253    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/26a709fe-0870-4529-905c-5075b7758948-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "26a709fe-0870-4529-905c-5075b7758948" (UID: "26a709fe-0870-4529-905c-5075b7758948"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:18:13.545585 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.545555    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-kube-api-access-6ztrv" (OuterVolumeSpecName: "kube-api-access-6ztrv") pod "26a709fe-0870-4529-905c-5075b7758948" (UID: "26a709fe-0870-4529-905c-5075b7758948"). InnerVolumeSpecName "kube-api-access-6ztrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:13.545674 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.545607    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-istio-token" (OuterVolumeSpecName: "istio-token") pod "26a709fe-0870-4529-905c-5075b7758948" (UID: "26a709fe-0870-4529-905c-5075b7758948"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:13.545713 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.545686    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "26a709fe-0870-4529-905c-5075b7758948" (UID: "26a709fe-0870-4529-905c-5075b7758948"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:13.644502 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.644467    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26a709fe-0870-4529-905c-5075b7758948-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:13.644502 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.644496    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:13.644502 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.644505    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:13.644502 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.644513    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26a709fe-0870-4529-905c-5075b7758948-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:13.644852 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.644521    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:13.644852 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.644528    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26a709fe-0870-4529-905c-5075b7758948-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:13.644852 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.644537    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ztrv\" (UniqueName: \"kubernetes.io/projected/26a709fe-0870-4529-905c-5075b7758948-kube-api-access-6ztrv\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:13.946930 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.946831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.949270 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.949241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:13.953068 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.953042    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"]
Sep 07 09:18:13.953463 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.953450    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy"
Sep 07 09:18:13.953505 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.953465    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy"
Sep 07 09:18:13.953537 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.953532    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy"
Sep 07 09:18:13.953617 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.953540    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy"
Sep 07 09:18:13.955461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.955445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:13.958625 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.958600    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-263421\""
Sep 07 09:18:13.958867 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.958848    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-263421-dockercfg-x6kmk\""
Sep 07 09:18:13.969313 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:13.969274    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"]
Sep 07 09:18:14.002301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.002248    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"]
Sep 07 09:18:14.002673 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.002660    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy"
Sep 07 09:18:14.002715 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.002675    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a709fe-0870-4529-905c-5075b7758948" containerName="istio-proxy"
Sep 07 09:18:14.004773 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.004755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:14.016578 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.016553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"]
Sep 07 09:18:14.048430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.048386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv\" (UID: \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:14.048612 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.048454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfsrv\" (UniqueName: \"kubernetes.io/projected/ad152bed-9809-4a9b-bc47-67a82cfd0908-kube-api-access-jfsrv\") pod \"payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j\" (UID: \"ad152bed-9809-4a9b-bc47-67a82cfd0908\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:14.048612 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.048491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad152bed-9809-4a9b-bc47-67a82cfd0908-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j\" (UID: \"ad152bed-9809-4a9b-bc47-67a82cfd0908\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:14.048744 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.048625    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5dc5\" (UniqueName: \"kubernetes.io/projected/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-kube-api-access-h5dc5\") pod \"payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv\" (UID: \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:14.149467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.149382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5dc5\" (UniqueName: \"kubernetes.io/projected/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-kube-api-access-h5dc5\") pod \"payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv\" (UID: \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:14.149739 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.149672    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv\" (UID: \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:14.149964 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.149844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfsrv\" (UniqueName: \"kubernetes.io/projected/ad152bed-9809-4a9b-bc47-67a82cfd0908-kube-api-access-jfsrv\") pod \"payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j\" (UID: \"ad152bed-9809-4a9b-bc47-67a82cfd0908\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:14.149964 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.149878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad152bed-9809-4a9b-bc47-67a82cfd0908-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j\" (UID: \"ad152bed-9809-4a9b-bc47-67a82cfd0908\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:14.150639 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.150616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv\" (UID: \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:14.150716 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.150670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad152bed-9809-4a9b-bc47-67a82cfd0908-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j\" (UID: \"ad152bed-9809-4a9b-bc47-67a82cfd0908\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:14.158070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.158041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5dc5\" (UniqueName: \"kubernetes.io/projected/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-kube-api-access-h5dc5\") pod \"payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv\" (UID: \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:14.158070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.158054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfsrv\" (UniqueName: \"kubernetes.io/projected/ad152bed-9809-4a9b-bc47-67a82cfd0908-kube-api-access-jfsrv\") pod \"payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j\" (UID: \"ad152bed-9809-4a9b-bc47-67a82cfd0908\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:14.170592 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.170539    2563 generic.go:356] "Generic (PLEG): container finished" podID="26a709fe-0870-4529-905c-5075b7758948" containerID="085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba" exitCode=0
Sep 07 09:18:14.170714 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.170602    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"
Sep 07 09:18:14.170714 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.170635    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" event={"ID":"26a709fe-0870-4529-905c-5075b7758948","Type":"ContainerDied","Data":"085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba"}
Sep 07 09:18:14.170714 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.170692    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4" event={"ID":"26a709fe-0870-4529-905c-5075b7758948","Type":"ContainerDied","Data":"a1798012d6be10482a180e712f10e5e89787c99d37dd914b65dba09253ec3804"}
Sep 07 09:18:14.170823 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.170727    2563 scope.go:117] "RemoveContainer" containerID="085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba"
Sep 07 09:18:14.179772 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.179752    2563 scope.go:117] "RemoveContainer" containerID="29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084"
Sep 07 09:18:14.187577 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.187560    2563 scope.go:117] "RemoveContainer" containerID="085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba"
Sep 07 09:18:14.188185 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:18:14.188155    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba\": container with ID starting with 085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba not found: ID does not exist" containerID="085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba"
Sep 07 09:18:14.188304 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.188186    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba"} err="failed to get container status \"085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba\": rpc error: code = NotFound desc = could not find container \"085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba\": container with ID starting with 085a9091dfec3c3e0820cf9ace93f5b547dbcf9ae7e93423d3f706800f38cbba not found: ID does not exist"
Sep 07 09:18:14.188304 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.188204    2563 scope.go:117] "RemoveContainer" containerID="29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084"
Sep 07 09:18:14.188541 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:18:14.188521    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084\": container with ID starting with 29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084 not found: ID does not exist" containerID="29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084"
Sep 07 09:18:14.188583 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.188547    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084"} err="failed to get container status \"29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084\": rpc error: code = NotFound desc = could not find container \"29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084\": container with ID starting with 29656cbbc51c0451627e77f8cced57587e3faaf66d6118e8d2b4dcc026ec1084 not found: ID does not exist"
Sep 07 09:18:14.194160 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.194131    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"]
Sep 07 09:18:14.198665 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.198594    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-614a02-openshift-default-746865fc5d-gg2h4"]
Sep 07 09:18:14.225103 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.225071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:14.266762 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.266613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:14.314957 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.314882    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:14.472842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.472749    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26a709fe-0870-4529-905c-5075b7758948" path="/var/lib/kubelet/pods/26a709fe-0870-4529-905c-5075b7758948/volumes"
Sep 07 09:18:14.503998 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.503969    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"]
Sep 07 09:18:14.506015 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:18:14.505979    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de4ac61_5425_4f69_ab7a_ca8e319579eb.slice/crio-b4bc52b3bcbc1ae8050381a6cf05c4849b2d72c56377b81cce65a17fd6c22a9f WatchSource:0}: Error finding container b4bc52b3bcbc1ae8050381a6cf05c4849b2d72c56377b81cce65a17fd6c22a9f: Status 404 returned error can't find the container with id b4bc52b3bcbc1ae8050381a6cf05c4849b2d72c56377b81cce65a17fd6c22a9f
Sep 07 09:18:14.507411 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.507393    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:18:14.550240 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.550207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"]
Sep 07 09:18:14.551980 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:18:14.551946    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad152bed_9809_4a9b_bc47_67a82cfd0908.slice/crio-06c4c10bd681f569da5f2fabab0ca6758c17538c3335e57dcc2d7f8d44f29758 WatchSource:0}: Error finding container 06c4c10bd681f569da5f2fabab0ca6758c17538c3335e57dcc2d7f8d44f29758: Status 404 returned error can't find the container with id 06c4c10bd681f569da5f2fabab0ca6758c17538c3335e57dcc2d7f8d44f29758
Sep 07 09:18:14.614055 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:14.614028    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"]
Sep 07 09:18:14.616522 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:18:14.616492    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3182cfb1_ff92_4a63_9ff5_7b01eb68a48b.slice/crio-68e5f003cd5cebcde2b91836d0c906ffb13bf4aa836e0b004989475293464c01 WatchSource:0}: Error finding container 68e5f003cd5cebcde2b91836d0c906ffb13bf4aa836e0b004989475293464c01: Status 404 returned error can't find the container with id 68e5f003cd5cebcde2b91836d0c906ffb13bf4aa836e0b004989475293464c01
Sep 07 09:18:15.177393 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.177353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv" event={"ID":"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b","Type":"ContainerStarted","Data":"c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2"}
Sep 07 09:18:15.177648 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.177421    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv" event={"ID":"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b","Type":"ContainerStarted","Data":"68e5f003cd5cebcde2b91836d0c906ffb13bf4aa836e0b004989475293464c01"}
Sep 07 09:18:15.177648 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.177480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:15.179235 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.179203    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6" event={"ID":"5de4ac61-5425-4f69-ab7a-ca8e319579eb","Type":"ContainerStarted","Data":"a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0"}
Sep 07 09:18:15.179368 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.179255    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6" event={"ID":"5de4ac61-5425-4f69-ab7a-ca8e319579eb","Type":"ContainerStarted","Data":"b4bc52b3bcbc1ae8050381a6cf05c4849b2d72c56377b81cce65a17fd6c22a9f"}
Sep 07 09:18:15.179368 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.179322    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:15.182163 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.182135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j" event={"ID":"ad152bed-9809-4a9b-bc47-67a82cfd0908","Type":"ContainerStarted","Data":"ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601"}
Sep 07 09:18:15.182309 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.182179    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j" event={"ID":"ad152bed-9809-4a9b-bc47-67a82cfd0908","Type":"ContainerStarted","Data":"06c4c10bd681f569da5f2fabab0ca6758c17538c3335e57dcc2d7f8d44f29758"}
Sep 07 09:18:15.182309 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.182229    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:15.194595 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.194500    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv" podStartSLOduration=2.194485024 podStartE2EDuration="2.194485024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:15.19300676 +0000 UTC m=+1153.162177366" watchObservedRunningTime="2026-09-07 09:18:15.194485024 +0000 UTC m=+1153.163655629"
Sep 07 09:18:15.212063 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.212005    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j" podStartSLOduration=2.211990045 podStartE2EDuration="2.211990045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:15.211972102 +0000 UTC m=+1153.181142708" watchObservedRunningTime="2026-09-07 09:18:15.211990045 +0000 UTC m=+1153.181160650"
Sep 07 09:18:15.233600 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:15.233542    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6" podStartSLOduration=1.797626595 podStartE2EDuration="2.233523859s" totalImagesPullingTime="435.897264ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:15.231180652 +0000 UTC m=+1153.200351257" watchObservedRunningTime="2026-09-07 09:18:15.233523859 +0000 UTC m=+1153.202694463"
Sep 07 09:18:19.384501 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:19.384465    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"]
Sep 07 09:18:19.385366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:19.385330    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8" podUID="4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2" containerName="payload-pre-processing" containerID="cri-o://caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87" gracePeriod=30
Sep 07 09:18:19.680634 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:19.680603    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:18:19.806739 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:19.806702    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-plugins-config-volume\") pod \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\" (UID: \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\") "
Sep 07 09:18:19.806909 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:19.806773    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv4dh\" (UniqueName: \"kubernetes.io/projected/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-kube-api-access-sv4dh\") pod \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\" (UID: \"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2\") "
Sep 07 09:18:19.807046 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:19.807020    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2" (UID: "4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:18:19.808881 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:19.808860    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-kube-api-access-sv4dh" (OuterVolumeSpecName: "kube-api-access-sv4dh") pod "4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2" (UID: "4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2"). InnerVolumeSpecName "kube-api-access-sv4dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:19.908496 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:19.908397    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:19.908496 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:19.908429    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv4dh\" (UniqueName: \"kubernetes.io/projected/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2-kube-api-access-sv4dh\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:18:20.207626 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.207530    2563 generic.go:356] "Generic (PLEG): container finished" podID="4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2" containerID="caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87" exitCode=0
Sep 07 09:18:20.207800 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.207621    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"
Sep 07 09:18:20.207800 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.207621    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8" event={"ID":"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2","Type":"ContainerDied","Data":"caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87"}
Sep 07 09:18:20.207800 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.207759    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8" event={"ID":"4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2","Type":"ContainerDied","Data":"f6127f97bc9835b30d80e892e8ca1d336a9699f0d39994beefd7a044f7abfb79"}
Sep 07 09:18:20.207800 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.207785    2563 scope.go:117] "RemoveContainer" containerID="caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87"
Sep 07 09:18:20.226892 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.226869    2563 scope.go:117] "RemoveContainer" containerID="caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87"
Sep 07 09:18:20.227321 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:18:20.227299    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87\": container with ID starting with caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87 not found: ID does not exist" containerID="caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87"
Sep 07 09:18:20.227414 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.227332    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87"} err="failed to get container status \"caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87\": rpc error: code = NotFound desc = could not find container \"caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87\": container with ID starting with caf3d90c09237e09402d1ad4ebe692ed1bce2e62243ca9e9d6635c53c2ec5d87 not found: ID does not exist"
Sep 07 09:18:20.233787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.233763    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"]
Sep 07 09:18:20.239997 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.239968    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3bcf58-7dbb55574b-xhkl8"]
Sep 07 09:18:20.470703 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:20.470610    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2" path="/var/lib/kubelet/pods/4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2/volumes"
Sep 07 09:18:21.192578 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:21.192551    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:18:26.189111 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.189070    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:18:26.189562 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.189154    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:18:26.774840 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.774805    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"]
Sep 07 09:18:26.775224 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.775209    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2" containerName="payload-pre-processing"
Sep 07 09:18:26.775312 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.775226    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2" containerName="payload-pre-processing"
Sep 07 09:18:26.775363 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.775335    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cd29d14-a61d-4e3f-9fd9-db45a4a6f0b2" containerName="payload-pre-processing"
Sep 07 09:18:26.778038 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.778007    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.780684 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.780654    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-3dc031-openshift-default-dockercfg-j86qk\""
Sep 07 09:18:26.791756 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.791730    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"]
Sep 07 09:18:26.875240 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.875193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-envoy\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.875240 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.875242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh6td\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-kube-api-access-wh6td\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.875494 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.875270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-credential-socket\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.875494 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.875317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istiod-ca-cert\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.875494 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.875353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-data\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.875494 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.875382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-podinfo\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.875494 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.875428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-token\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.875732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.875489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-socket\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.875732 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.875594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-certs\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.976699 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.976658    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-credential-socket\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.976699 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.976699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istiod-ca-cert\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.976977 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.976722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-data\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.976977 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.976745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-podinfo\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.976977 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.976807    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-token\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.976977 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.976831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-socket\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.976977 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.976875    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-certs\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.976977 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.976939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-envoy\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.976977 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.976966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh6td\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-kube-api-access-wh6td\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.977387 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.977154    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-credential-socket\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.977480 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.977456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-certs\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.977674 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.977648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istiod-ca-cert\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.977756 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.977707    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-socket\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.977822 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.977761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-data\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.979398 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.979376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-envoy\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.979944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.979924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-podinfo\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.985010 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.984979    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-token\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:26.985159 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:26.985138    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh6td\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-kube-api-access-wh6td\") pod \"e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") " pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:27.090043 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:27.089959    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:27.197530 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:27.197475    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 09:18:27.198015 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:27.197559    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 09:18:27.365228 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:27.365198    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"]
Sep 07 09:18:27.367793 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:18:27.367757    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e1b4a07_b6b5_47bd_b0e1_dbd79e9f19b0.slice/crio-af5d208239cfb5fda71cc2ab5fe5a2c016a7827921988de2f4084f115c22bf87 WatchSource:0}: Error finding container af5d208239cfb5fda71cc2ab5fe5a2c016a7827921988de2f4084f115c22bf87: Status 404 returned error can't find the container with id af5d208239cfb5fda71cc2ab5fe5a2c016a7827921988de2f4084f115c22bf87
Sep 07 09:18:27.370102 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:27.370069    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:27.370204 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:27.370150    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:27.370204 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:27.370194    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:28.256085 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:28.256051    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" event={"ID":"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0","Type":"ContainerStarted","Data":"768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f"}
Sep 07 09:18:28.256652 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:28.256110    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" event={"ID":"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0","Type":"ContainerStarted","Data":"af5d208239cfb5fda71cc2ab5fe5a2c016a7827921988de2f4084f115c22bf87"}
Sep 07 09:18:28.257960 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:28.257931    2563 generic.go:356] "Generic (PLEG): container finished" podID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerID="8559e36e712cbc561a6aead535392755ff300984770025edff4e71900041f723" exitCode=0
Sep 07 09:18:28.258071 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:28.257992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" event={"ID":"308afb0b-413a-46ce-bbd4-9ef6d000049d","Type":"ContainerDied","Data":"8559e36e712cbc561a6aead535392755ff300984770025edff4e71900041f723"}
Sep 07 09:18:28.258402 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:28.258387    2563 scope.go:117] "RemoveContainer" containerID="8559e36e712cbc561a6aead535392755ff300984770025edff4e71900041f723"
Sep 07 09:18:28.259482 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:28.259455    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:28.259562 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:28.259521    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:28.259562 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:28.259553    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:28.276988 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:28.276911    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" podStartSLOduration=2.276889035 podStartE2EDuration="2.276889035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:28.274101424 +0000 UTC m=+1166.243272028" watchObservedRunningTime="2026-09-07 09:18:28.276889035 +0000 UTC m=+1166.246059638"
Sep 07 09:18:29.090537 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:29.090497    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:29.095795 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:29.095768    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:29.270223 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:29.270188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" event={"ID":"308afb0b-413a-46ce-bbd4-9ef6d000049d","Type":"ContainerStarted","Data":"1e7057a92584d2147833efd32df085f5473ce0501a3e68e068dae5a9abb4bef9"}
Sep 07 09:18:29.271250 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:29.270945    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:29.271997 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:29.271972    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:30.014762 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:30.014716    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:30.015052 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:30.015022    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 07 09:18:30.015109 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:30.015087    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 07 09:18:31.015441 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:31.015399    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 07 09:18:31.015834 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:31.015476    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 07 09:18:32.014702 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:32.014670    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 07 09:18:32.014868 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:32.014738    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 07 09:18:33.015096 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:33.015045    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 07 09:18:33.015535 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:33.015134    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 07 09:18:34.015311 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:34.015244    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 07 09:18:34.015734 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:34.015342    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 07 09:18:35.096885 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:35.096849    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:35.097470 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:35.097243    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:35.098001 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:35.097981    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:18:43.072535 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.072488    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"]
Sep 07 09:18:43.075336 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.075313    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:43.078090 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.078068    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-5560ef-dockercfg-zlmzk\""
Sep 07 09:18:43.078209 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.078077    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-5560ef\""
Sep 07 09:18:43.086518 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.086488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"]
Sep 07 09:18:43.145799 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.145764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b621e1b-28cd-4a07-af8a-b876ea399907-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl\" (UID: \"0b621e1b-28cd-4a07-af8a-b876ea399907\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:43.145995 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.145822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zjpj\" (UniqueName: \"kubernetes.io/projected/0b621e1b-28cd-4a07-af8a-b876ea399907-kube-api-access-9zjpj\") pod \"payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl\" (UID: \"0b621e1b-28cd-4a07-af8a-b876ea399907\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:43.246580 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.246528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zjpj\" (UniqueName: \"kubernetes.io/projected/0b621e1b-28cd-4a07-af8a-b876ea399907-kube-api-access-9zjpj\") pod \"payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl\" (UID: \"0b621e1b-28cd-4a07-af8a-b876ea399907\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:43.246773 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.246673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b621e1b-28cd-4a07-af8a-b876ea399907-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl\" (UID: \"0b621e1b-28cd-4a07-af8a-b876ea399907\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:43.247392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.247373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b621e1b-28cd-4a07-af8a-b876ea399907-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl\" (UID: \"0b621e1b-28cd-4a07-af8a-b876ea399907\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:43.254918 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.254893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zjpj\" (UniqueName: \"kubernetes.io/projected/0b621e1b-28cd-4a07-af8a-b876ea399907-kube-api-access-9zjpj\") pod \"payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl\" (UID: \"0b621e1b-28cd-4a07-af8a-b876ea399907\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:43.387263 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.387163    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:43.665122 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:43.665094    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"]
Sep 07 09:18:43.692307 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:18:43.692257    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b621e1b_28cd_4a07_af8a_b876ea399907.slice/crio-a4e98fcb6d3377a4f0fb3612ecc757bd709c6974f2573452039805426e121401 WatchSource:0}: Error finding container a4e98fcb6d3377a4f0fb3612ecc757bd709c6974f2573452039805426e121401: Status 404 returned error can't find the container with id a4e98fcb6d3377a4f0fb3612ecc757bd709c6974f2573452039805426e121401
Sep 07 09:18:44.272906 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.272868    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 07 09:18:44.273313 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.272923    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 07 09:18:44.350941 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.350900    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerID="768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f" exitCode=0
Sep 07 09:18:44.351148 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.350968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" event={"ID":"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0","Type":"ContainerDied","Data":"768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f"}
Sep 07 09:18:44.351500 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.351478    2563 scope.go:117] "RemoveContainer" containerID="768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f"
Sep 07 09:18:44.352681 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.352647    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:44.352802 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.352739    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:44.352802 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.352776    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:44.352925 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.352803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl" event={"ID":"0b621e1b-28cd-4a07-af8a-b876ea399907","Type":"ContainerStarted","Data":"9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e"}
Sep 07 09:18:44.352925 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.352857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl" event={"ID":"0b621e1b-28cd-4a07-af8a-b876ea399907","Type":"ContainerStarted","Data":"a4e98fcb6d3377a4f0fb3612ecc757bd709c6974f2573452039805426e121401"}
Sep 07 09:18:44.352925 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.352906    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:44.390981 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:44.390928    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl" podStartSLOduration=1.390908181 podStartE2EDuration="1.390908181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:44.389672515 +0000 UTC m=+1182.358843121" watchObservedRunningTime="2026-09-07 09:18:44.390908181 +0000 UTC m=+1182.360078788"
Sep 07 09:18:45.365995 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:45.365953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" event={"ID":"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0","Type":"ContainerStarted","Data":"94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669"}
Sep 07 09:18:46.090588 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:46.090539    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:46.090804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:46.090773    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 07 09:18:46.090885 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:46.090838    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 07 09:18:47.090550 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:47.090510    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 07 09:18:47.090964 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:47.090569    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 07 09:18:48.091267 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:48.091219    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 07 09:18:48.091815 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:48.091325    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 07 09:18:49.091106 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:49.091063    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 07 09:18:49.091317 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:49.091129    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 07 09:18:50.090706 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:50.090664    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 07 09:18:50.090915 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:50.090726    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 07 09:18:51.094994 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:51.094961    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:51.095510 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:51.095227    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:51.095890 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:51.095868    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:18:55.368118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.368079    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:18:55.794717 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.794674    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"]
Sep 07 09:18:55.798106 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.798079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.800809 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.800779    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-34de95-openshift-default-dockercfg-tcl82\""
Sep 07 09:18:55.809627 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.809602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"]
Sep 07 09:18:55.872088 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.872051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-credential-socket\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.872088 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.872092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-socket\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.872392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.872112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-data\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.872392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.872156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istiod-ca-cert\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.872392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.872220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-podinfo\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.872392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.872261    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-token\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.872392 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.872365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-certs\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.872582 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.872399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-envoy\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.872582 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.872430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hvj4\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-kube-api-access-8hvj4\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.973506 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-credential-socket\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.973697 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973515    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-socket\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.973697 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-data\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.973697 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973644    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istiod-ca-cert\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.973869 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-podinfo\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.973869 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-token\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.973869 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-certs\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.974040 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973903    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-credential-socket\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.974040 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-envoy\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.974040 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-data\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.974040 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.973995    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-socket\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.974304 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.974049    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hvj4\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-kube-api-access-8hvj4\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.974304 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.974175    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-certs\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.974459 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.974437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istiod-ca-cert\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.976016 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.975989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-envoy\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.976219 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.976199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-podinfo\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.981889 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.981864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-token\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:55.982024 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:55.981990    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hvj4\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-kube-api-access-8hvj4\") pod \"e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") " pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:56.109069 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:56.108971    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:56.382545 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:56.382522    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"]
Sep 07 09:18:56.384802 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:18:56.384773    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod950b0191_3908_4fa5_8dc9_82f2a7a1d823.slice/crio-5e49d6613144928cb6447a0c3450e0f78394aa04ebd1849197ef5afc8c9d65cf WatchSource:0}: Error finding container 5e49d6613144928cb6447a0c3450e0f78394aa04ebd1849197ef5afc8c9d65cf: Status 404 returned error can't find the container with id 5e49d6613144928cb6447a0c3450e0f78394aa04ebd1849197ef5afc8c9d65cf
Sep 07 09:18:56.386991 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:56.386957    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:56.387056 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:56.387032    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:56.387094 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:56.387073    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:18:56.430482 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:56.430449    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" event={"ID":"950b0191-3908-4fa5-8dc9-82f2a7a1d823","Type":"ContainerStarted","Data":"5e49d6613144928cb6447a0c3450e0f78394aa04ebd1849197ef5afc8c9d65cf"}
Sep 07 09:18:57.436575 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:57.436539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" event={"ID":"950b0191-3908-4fa5-8dc9-82f2a7a1d823","Type":"ContainerStarted","Data":"88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e"}
Sep 07 09:18:57.458327 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:57.458250    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" podStartSLOduration=2.45823199 podStartE2EDuration="2.45823199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:57.456270553 +0000 UTC m=+1195.425441159" watchObservedRunningTime="2026-09-07 09:18:57.45823199 +0000 UTC m=+1195.427402602"
Sep 07 09:18:58.109865 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:58.109820    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:58.115595 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:58.115566    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:58.441679 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:58.441601    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:18:58.442583 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:18:58.442554    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:19:01.279475 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.279433    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"]
Sep 07 09:19:01.282312 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.282262    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:01.284797 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.284773    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-34de95\""
Sep 07 09:19:01.284940 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.284843    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-34de95-dockercfg-mnl2w\""
Sep 07 09:19:01.293421 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.293390    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"]
Sep 07 09:19:01.329507 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.329472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/836c06d2-3ed8-4c0f-b681-18768ffa6221-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4\" (UID: \"836c06d2-3ed8-4c0f-b681-18768ffa6221\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:01.329674 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.329599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksf2l\" (UniqueName: \"kubernetes.io/projected/836c06d2-3ed8-4c0f-b681-18768ffa6221-kube-api-access-ksf2l\") pod \"payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4\" (UID: \"836c06d2-3ed8-4c0f-b681-18768ffa6221\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:01.430612 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.430573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/836c06d2-3ed8-4c0f-b681-18768ffa6221-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4\" (UID: \"836c06d2-3ed8-4c0f-b681-18768ffa6221\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:01.430906 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.430710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksf2l\" (UniqueName: \"kubernetes.io/projected/836c06d2-3ed8-4c0f-b681-18768ffa6221-kube-api-access-ksf2l\") pod \"payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4\" (UID: \"836c06d2-3ed8-4c0f-b681-18768ffa6221\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:01.431358 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.431339    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/836c06d2-3ed8-4c0f-b681-18768ffa6221-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4\" (UID: \"836c06d2-3ed8-4c0f-b681-18768ffa6221\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:01.439622 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.439594    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksf2l\" (UniqueName: \"kubernetes.io/projected/836c06d2-3ed8-4c0f-b681-18768ffa6221-kube-api-access-ksf2l\") pod \"payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4\" (UID: \"836c06d2-3ed8-4c0f-b681-18768ffa6221\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:01.595148 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.595052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:01.870671 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:01.870644    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"]
Sep 07 09:19:01.872539 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:19:01.872508    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod836c06d2_3ed8_4c0f_b681_18768ffa6221.slice/crio-b204d2099b01a5484428d0d2be504702449211a29f00ba270e9450637be45407 WatchSource:0}: Error finding container b204d2099b01a5484428d0d2be504702449211a29f00ba270e9450637be45407: Status 404 returned error can't find the container with id b204d2099b01a5484428d0d2be504702449211a29f00ba270e9450637be45407
Sep 07 09:19:02.465752 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:02.465715    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4" event={"ID":"836c06d2-3ed8-4c0f-b681-18768ffa6221","Type":"ContainerStarted","Data":"c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9"}
Sep 07 09:19:02.466208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:02.465768    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4" event={"ID":"836c06d2-3ed8-4c0f-b681-18768ffa6221","Type":"ContainerStarted","Data":"b204d2099b01a5484428d0d2be504702449211a29f00ba270e9450637be45407"}
Sep 07 09:19:02.466208 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:02.465787    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:02.483975 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:02.483923    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4" podStartSLOduration=1.483906362 podStartE2EDuration="1.483906362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:02.481838636 +0000 UTC m=+1200.451009244" watchObservedRunningTime="2026-09-07 09:19:02.483906362 +0000 UTC m=+1200.453076967"
Sep 07 09:19:04.970632 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:04.970599    2563 scope.go:117] "RemoveContainer" containerID="a3c419bfa721431b6f792d94c7633f72214842039a47b3c4d7066871a14f6859"
Sep 07 09:19:04.979866 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:04.979836    2563 scope.go:117] "RemoveContainer" containerID="8741093d0870aacbb6359381638f69b4b2eec600eb0f0ccfc524508ea27df8bd"
Sep 07 09:19:06.956047 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:06.956012    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:06.959411 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:06.959389    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:06.962112 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:06.962088    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ckkfv\""
Sep 07 09:19:06.965769 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:06.965742    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:07.087314 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:07.087240    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2t4f\" (UniqueName: \"kubernetes.io/projected/818013a4-3d67-4950-9c3d-9626887030c4-kube-api-access-g2t4f\") pod \"test-curl-7981-139865525441136\" (UID: \"818013a4-3d67-4950-9c3d-9626887030c4\") " pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:07.187927 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:07.187881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2t4f\" (UniqueName: \"kubernetes.io/projected/818013a4-3d67-4950-9c3d-9626887030c4-kube-api-access-g2t4f\") pod \"test-curl-7981-139865525441136\" (UID: \"818013a4-3d67-4950-9c3d-9626887030c4\") " pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:07.195817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:07.195789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2t4f\" (UniqueName: \"kubernetes.io/projected/818013a4-3d67-4950-9c3d-9626887030c4-kube-api-access-g2t4f\") pod \"test-curl-7981-139865525441136\" (UID: \"818013a4-3d67-4950-9c3d-9626887030c4\") " pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:07.270941 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:07.270897    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:07.539546 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:07.539511    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:07.542109 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:19:07.542076    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod818013a4_3d67_4950_9c3d_9626887030c4.slice/crio-676084a62f371b9d153806192dbff08348bfc213137e5a2599c3aa268925cda2 WatchSource:0}: Error finding container 676084a62f371b9d153806192dbff08348bfc213137e5a2599c3aa268925cda2: Status 404 returned error can't find the container with id 676084a62f371b9d153806192dbff08348bfc213137e5a2599c3aa268925cda2
Sep 07 09:19:08.497423 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:08.497384    2563 generic.go:356] "Generic (PLEG): container finished" podID="818013a4-3d67-4950-9c3d-9626887030c4" containerID="9e3e662350d6aac24d30395b245b19b214e0616e38bc44c4bd011a044a0691ac" exitCode=0
Sep 07 09:19:08.497826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:08.497444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7981-139865525441136" event={"ID":"818013a4-3d67-4950-9c3d-9626887030c4","Type":"ContainerDied","Data":"9e3e662350d6aac24d30395b245b19b214e0616e38bc44c4bd011a044a0691ac"}
Sep 07 09:19:08.497826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:08.497472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7981-139865525441136" event={"ID":"818013a4-3d67-4950-9c3d-9626887030c4","Type":"ContainerStarted","Data":"676084a62f371b9d153806192dbff08348bfc213137e5a2599c3aa268925cda2"}
Sep 07 09:19:09.679569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.679542    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:09.715361 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.715331    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7981-139865525441136_818013a4-3d67-4950-9c3d-9626887030c4/test-curl-7981-139865525441136/0.log"
Sep 07 09:19:09.740080 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.740043    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:09.745403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.745358    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:09.815701 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.815607    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2t4f\" (UniqueName: \"kubernetes.io/projected/818013a4-3d67-4950-9c3d-9626887030c4-kube-api-access-g2t4f\") pod \"818013a4-3d67-4950-9c3d-9626887030c4\" (UID: \"818013a4-3d67-4950-9c3d-9626887030c4\") "
Sep 07 09:19:09.817765 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.817736    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818013a4-3d67-4950-9c3d-9626887030c4-kube-api-access-g2t4f" (OuterVolumeSpecName: "kube-api-access-g2t4f") pod "818013a4-3d67-4950-9c3d-9626887030c4" (UID: "818013a4-3d67-4950-9c3d-9626887030c4"). InnerVolumeSpecName "kube-api-access-g2t4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:19:09.888871 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.888834    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:09.889265 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.889251    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818013a4-3d67-4950-9c3d-9626887030c4" containerName="test-curl-7981-139865525441136"
Sep 07 09:19:09.889356 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.889267    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="818013a4-3d67-4950-9c3d-9626887030c4" containerName="test-curl-7981-139865525441136"
Sep 07 09:19:09.889356 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.889346    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="818013a4-3d67-4950-9c3d-9626887030c4" containerName="test-curl-7981-139865525441136"
Sep 07 09:19:09.891358 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.891341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:09.899028 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.898944    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:09.916961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:09.916927    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2t4f\" (UniqueName: \"kubernetes.io/projected/818013a4-3d67-4950-9c3d-9626887030c4-kube-api-access-g2t4f\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:19:10.018114 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:10.018072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72sck\" (UniqueName: \"kubernetes.io/projected/3b82c0cf-387c-4dbb-aed4-deb493933d0b-kube-api-access-72sck\") pod \"test-curl-7981-139865525441136\" (UID: \"3b82c0cf-387c-4dbb-aed4-deb493933d0b\") " pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:10.118933 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:10.118836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72sck\" (UniqueName: \"kubernetes.io/projected/3b82c0cf-387c-4dbb-aed4-deb493933d0b-kube-api-access-72sck\") pod \"test-curl-7981-139865525441136\" (UID: \"3b82c0cf-387c-4dbb-aed4-deb493933d0b\") " pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:10.126963 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:10.126935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72sck\" (UniqueName: \"kubernetes.io/projected/3b82c0cf-387c-4dbb-aed4-deb493933d0b-kube-api-access-72sck\") pod \"test-curl-7981-139865525441136\" (UID: \"3b82c0cf-387c-4dbb-aed4-deb493933d0b\") " pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:10.201724 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:10.201689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:10.468910 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:10.468809    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="818013a4-3d67-4950-9c3d-9626887030c4" path="/var/lib/kubelet/pods/818013a4-3d67-4950-9c3d-9626887030c4/volumes"
Sep 07 09:19:10.492501 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:10.492474    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:10.494866 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:19:10.494836    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b82c0cf_387c_4dbb_aed4_deb493933d0b.slice/crio-8a3d67cc7715312d94fa8812b12c111be37de6517f68e1d8a851fe196468e644 WatchSource:0}: Error finding container 8a3d67cc7715312d94fa8812b12c111be37de6517f68e1d8a851fe196468e644: Status 404 returned error can't find the container with id 8a3d67cc7715312d94fa8812b12c111be37de6517f68e1d8a851fe196468e644
Sep 07 09:19:10.509726 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:10.509695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7981-139865525441136" event={"ID":"3b82c0cf-387c-4dbb-aed4-deb493933d0b","Type":"ContainerStarted","Data":"8a3d67cc7715312d94fa8812b12c111be37de6517f68e1d8a851fe196468e644"}
Sep 07 09:19:10.510942 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:10.510922    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:10.511063 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:10.510923    2563 scope.go:117] "RemoveContainer" containerID="9e3e662350d6aac24d30395b245b19b214e0616e38bc44c4bd011a044a0691ac"
Sep 07 09:19:11.518947 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:11.518913    2563 generic.go:356] "Generic (PLEG): container finished" podID="3b82c0cf-387c-4dbb-aed4-deb493933d0b" containerID="b59324cc807efbefd0b7ae516852f5d951ecd7642cf6051f34e9454b53c3e7bf" exitCode=0
Sep 07 09:19:11.519310 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:11.518950    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7981-139865525441136" event={"ID":"3b82c0cf-387c-4dbb-aed4-deb493933d0b","Type":"ContainerDied","Data":"b59324cc807efbefd0b7ae516852f5d951ecd7642cf6051f34e9454b53c3e7bf"}
Sep 07 09:19:12.670573 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.670549    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:12.690131 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.690085    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7981-139865525441136_3b82c0cf-387c-4dbb-aed4-deb493933d0b/test-curl-7981-139865525441136/0.log"
Sep 07 09:19:12.711303 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.711237    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:12.713711 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.713688    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7981-139865525441136"]
Sep 07 09:19:12.745129 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.745099    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72sck\" (UniqueName: \"kubernetes.io/projected/3b82c0cf-387c-4dbb-aed4-deb493933d0b-kube-api-access-72sck\") pod \"3b82c0cf-387c-4dbb-aed4-deb493933d0b\" (UID: \"3b82c0cf-387c-4dbb-aed4-deb493933d0b\") "
Sep 07 09:19:12.747579 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.747546    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b82c0cf-387c-4dbb-aed4-deb493933d0b-kube-api-access-72sck" (OuterVolumeSpecName: "kube-api-access-72sck") pod "3b82c0cf-387c-4dbb-aed4-deb493933d0b" (UID: "3b82c0cf-387c-4dbb-aed4-deb493933d0b"). InnerVolumeSpecName "kube-api-access-72sck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:19:12.839475 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.839386    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:12.839811 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.839795    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b82c0cf-387c-4dbb-aed4-deb493933d0b" containerName="test-curl-7981-139865525441136"
Sep 07 09:19:12.839811 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.839811    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b82c0cf-387c-4dbb-aed4-deb493933d0b" containerName="test-curl-7981-139865525441136"
Sep 07 09:19:12.839952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.839874    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b82c0cf-387c-4dbb-aed4-deb493933d0b" containerName="test-curl-7981-139865525441136"
Sep 07 09:19:12.841771 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.841751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:12.846668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.846641    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72sck\" (UniqueName: \"kubernetes.io/projected/3b82c0cf-387c-4dbb-aed4-deb493933d0b-kube-api-access-72sck\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:19:12.849501 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.849471    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:12.947493 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:12.947447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8x82\" (UniqueName: \"kubernetes.io/projected/52f6d9ce-b6d3-4666-a791-9020dfc900cc-kube-api-access-w8x82\") pod \"test-curl-7981-139865525441712\" (UID: \"52f6d9ce-b6d3-4666-a791-9020dfc900cc\") " pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:13.048013 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:13.047975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8x82\" (UniqueName: \"kubernetes.io/projected/52f6d9ce-b6d3-4666-a791-9020dfc900cc-kube-api-access-w8x82\") pod \"test-curl-7981-139865525441712\" (UID: \"52f6d9ce-b6d3-4666-a791-9020dfc900cc\") " pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:13.056332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:13.056304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8x82\" (UniqueName: \"kubernetes.io/projected/52f6d9ce-b6d3-4666-a791-9020dfc900cc-kube-api-access-w8x82\") pod \"test-curl-7981-139865525441712\" (UID: \"52f6d9ce-b6d3-4666-a791-9020dfc900cc\") " pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:13.153200 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:13.153113    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:13.433739 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:13.433710    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:13.437044 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:19:13.437012    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52f6d9ce_b6d3_4666_a791_9020dfc900cc.slice/crio-41e4010caf1f458b861e8168d77c97eea3b8c2828562a0dec08e306a046fe36b WatchSource:0}: Error finding container 41e4010caf1f458b861e8168d77c97eea3b8c2828562a0dec08e306a046fe36b: Status 404 returned error can't find the container with id 41e4010caf1f458b861e8168d77c97eea3b8c2828562a0dec08e306a046fe36b
Sep 07 09:19:13.472727 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:13.472699    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:19:13.531075 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:13.531004    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:13.531075 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:13.531028    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a3d67cc7715312d94fa8812b12c111be37de6517f68e1d8a851fe196468e644"
Sep 07 09:19:13.532972 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:13.532935    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7981-139865525441712" event={"ID":"52f6d9ce-b6d3-4666-a791-9020dfc900cc","Type":"ContainerStarted","Data":"41e4010caf1f458b861e8168d77c97eea3b8c2828562a0dec08e306a046fe36b"}
Sep 07 09:19:13.545996 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:19:13.545956    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7981-139865525441136\" is forbidden: User \"system:node:ip-10-0-138-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-157.ec2.internal' and this object" podUID="3b82c0cf-387c-4dbb-aed4-deb493933d0b" pod="opendatahub/test-curl-7981-139865525441136"
Sep 07 09:19:14.466175 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:14.466077    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b82c0cf-387c-4dbb-aed4-deb493933d0b" path="/var/lib/kubelet/pods/3b82c0cf-387c-4dbb-aed4-deb493933d0b/volumes"
Sep 07 09:19:14.483469 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:14.483429    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 09:19:14.483647 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:14.483497    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 09:19:14.541507 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:14.541473    2563 generic.go:356] "Generic (PLEG): container finished" podID="52f6d9ce-b6d3-4666-a791-9020dfc900cc" containerID="e429b805412e32f06336a3a993a834803b80e2c64939c48891af668ec62cb1f4" exitCode=0
Sep 07 09:19:14.541668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:14.541519    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7981-139865525441712" event={"ID":"52f6d9ce-b6d3-4666-a791-9020dfc900cc","Type":"ContainerDied","Data":"e429b805412e32f06336a3a993a834803b80e2c64939c48891af668ec62cb1f4"}
Sep 07 09:19:15.681869 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.681840    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:15.701407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.701371    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7981-139865525441712_52f6d9ce-b6d3-4666-a791-9020dfc900cc/test-curl-7981-139865525441712/0.log"
Sep 07 09:19:15.724039 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.723996    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:15.729626 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.729596    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:15.781020 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.780994    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8x82\" (UniqueName: \"kubernetes.io/projected/52f6d9ce-b6d3-4666-a791-9020dfc900cc-kube-api-access-w8x82\") pod \"52f6d9ce-b6d3-4666-a791-9020dfc900cc\" (UID: \"52f6d9ce-b6d3-4666-a791-9020dfc900cc\") "
Sep 07 09:19:15.782918 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.782897    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f6d9ce-b6d3-4666-a791-9020dfc900cc-kube-api-access-w8x82" (OuterVolumeSpecName: "kube-api-access-w8x82") pod "52f6d9ce-b6d3-4666-a791-9020dfc900cc" (UID: "52f6d9ce-b6d3-4666-a791-9020dfc900cc"). InnerVolumeSpecName "kube-api-access-w8x82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:19:15.855667 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.855627    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:15.856080 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.856066    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52f6d9ce-b6d3-4666-a791-9020dfc900cc" containerName="test-curl-7981-139865525441712"
Sep 07 09:19:15.856128 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.856082    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="52f6d9ce-b6d3-4666-a791-9020dfc900cc" containerName="test-curl-7981-139865525441712"
Sep 07 09:19:15.856167 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.856157    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="52f6d9ce-b6d3-4666-a791-9020dfc900cc" containerName="test-curl-7981-139865525441712"
Sep 07 09:19:15.858355 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.858339    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:15.862613 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.862584    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:15.881827 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.881789    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8x82\" (UniqueName: \"kubernetes.io/projected/52f6d9ce-b6d3-4666-a791-9020dfc900cc-kube-api-access-w8x82\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:19:15.982639 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:15.982549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68s5c\" (UniqueName: \"kubernetes.io/projected/e81d1990-1c87-4fc4-98f2-db28f4d89b5d-kube-api-access-68s5c\") pod \"test-curl-7981-139865525441712\" (UID: \"e81d1990-1c87-4fc4-98f2-db28f4d89b5d\") " pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:16.083657 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.083543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68s5c\" (UniqueName: \"kubernetes.io/projected/e81d1990-1c87-4fc4-98f2-db28f4d89b5d-kube-api-access-68s5c\") pod \"test-curl-7981-139865525441712\" (UID: \"e81d1990-1c87-4fc4-98f2-db28f4d89b5d\") " pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:16.092641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.092604    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68s5c\" (UniqueName: \"kubernetes.io/projected/e81d1990-1c87-4fc4-98f2-db28f4d89b5d-kube-api-access-68s5c\") pod \"test-curl-7981-139865525441712\" (UID: \"e81d1990-1c87-4fc4-98f2-db28f4d89b5d\") " pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:16.168602 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.168562    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:16.440807 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.440780    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:16.442943 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:19:16.442915    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode81d1990_1c87_4fc4_98f2_db28f4d89b5d.slice/crio-4d8b4a3123b4dbab744e54cd63c85abe0049628183c97f1dccfebd7271f1d805 WatchSource:0}: Error finding container 4d8b4a3123b4dbab744e54cd63c85abe0049628183c97f1dccfebd7271f1d805: Status 404 returned error can't find the container with id 4d8b4a3123b4dbab744e54cd63c85abe0049628183c97f1dccfebd7271f1d805
Sep 07 09:19:16.464878 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.464849    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52f6d9ce-b6d3-4666-a791-9020dfc900cc" path="/var/lib/kubelet/pods/52f6d9ce-b6d3-4666-a791-9020dfc900cc/volumes"
Sep 07 09:19:16.552634 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.552603    2563 generic.go:356] "Generic (PLEG): container finished" podID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerID="88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e" exitCode=0
Sep 07 09:19:16.552819 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.552688    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" event={"ID":"950b0191-3908-4fa5-8dc9-82f2a7a1d823","Type":"ContainerDied","Data":"88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e"}
Sep 07 09:19:16.553201 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.553184    2563 scope.go:117] "RemoveContainer" containerID="88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e"
Sep 07 09:19:16.554030 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.554005    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:16.554102 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.554008    2563 scope.go:117] "RemoveContainer" containerID="e429b805412e32f06336a3a993a834803b80e2c64939c48891af668ec62cb1f4"
Sep 07 09:19:16.554460 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.554420    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:19:16.554565 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.554510    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:19:16.554628 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.554583    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:19:16.555233 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:16.555212    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7981-139865525441712" event={"ID":"e81d1990-1c87-4fc4-98f2-db28f4d89b5d","Type":"ContainerStarted","Data":"4d8b4a3123b4dbab744e54cd63c85abe0049628183c97f1dccfebd7271f1d805"}
Sep 07 09:19:17.565170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:17.565127    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" event={"ID":"950b0191-3908-4fa5-8dc9-82f2a7a1d823","Type":"ContainerStarted","Data":"1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad"}
Sep 07 09:19:17.568961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:17.568924    2563 generic.go:356] "Generic (PLEG): container finished" podID="e81d1990-1c87-4fc4-98f2-db28f4d89b5d" containerID="04065c08f1a9c06a0cf83c5d2f5d4f01c474ddd278bd8a70d433d499ae457b85" exitCode=0
Sep 07 09:19:17.569119 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:17.568970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7981-139865525441712" event={"ID":"e81d1990-1c87-4fc4-98f2-db28f4d89b5d","Type":"ContainerDied","Data":"04065c08f1a9c06a0cf83c5d2f5d4f01c474ddd278bd8a70d433d499ae457b85"}
Sep 07 09:19:18.109855 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.109817    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:19:18.110054 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.110030    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 07 09:19:18.110117 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.110096    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 07 09:19:18.722788 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.722764    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:18.767077 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.766237    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7981-139865525441712_e81d1990-1c87-4fc4-98f2-db28f4d89b5d/test-curl-7981-139865525441712/0.log"
Sep 07 09:19:18.790822 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.790792    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:18.796240 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.796208    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7981-139865525441712"]
Sep 07 09:19:18.810436 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.810406    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68s5c\" (UniqueName: \"kubernetes.io/projected/e81d1990-1c87-4fc4-98f2-db28f4d89b5d-kube-api-access-68s5c\") pod \"e81d1990-1c87-4fc4-98f2-db28f4d89b5d\" (UID: \"e81d1990-1c87-4fc4-98f2-db28f4d89b5d\") "
Sep 07 09:19:18.812488 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.812457    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e81d1990-1c87-4fc4-98f2-db28f4d89b5d-kube-api-access-68s5c" (OuterVolumeSpecName: "kube-api-access-68s5c") pod "e81d1990-1c87-4fc4-98f2-db28f4d89b5d" (UID: "e81d1990-1c87-4fc4-98f2-db28f4d89b5d"). InnerVolumeSpecName "kube-api-access-68s5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:19:18.912037 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:18.911941    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68s5c\" (UniqueName: \"kubernetes.io/projected/e81d1990-1c87-4fc4-98f2-db28f4d89b5d-kube-api-access-68s5c\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:19:19.109976 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:19.109935    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 07 09:19:19.110151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:19.109993    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 07 09:19:19.580095 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:19.580062    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d8b4a3123b4dbab744e54cd63c85abe0049628183c97f1dccfebd7271f1d805"
Sep 07 09:19:19.580095 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:19.580070    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:19.595614 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:19:19.595582    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7981-139865525441712\" is forbidden: User \"system:node:ip-10-0-138-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-157.ec2.internal' and this object" podUID="e81d1990-1c87-4fc4-98f2-db28f4d89b5d" pod="opendatahub/test-curl-7981-139865525441712"
Sep 07 09:19:20.109656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:20.109616    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 07 09:19:20.110156 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:20.109686    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 07 09:19:20.468535 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:20.468443    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e81d1990-1c87-4fc4-98f2-db28f4d89b5d" path="/var/lib/kubelet/pods/e81d1990-1c87-4fc4-98f2-db28f4d89b5d/volumes"
Sep 07 09:19:21.109893 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:21.109855    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 07 09:19:21.110326 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:21.109931    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 07 09:19:22.109731 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:22.109696    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 07 09:19:22.109915 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:22.109755    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 07 09:19:23.112905 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:23.112874    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:19:23.113327 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:23.113303    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:19:23.114070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:23.114049    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:19:24.379980 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.379943    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"]
Sep 07 09:19:24.380387 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.380359    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e81d1990-1c87-4fc4-98f2-db28f4d89b5d" containerName="test-curl-7981-139865525441712"
Sep 07 09:19:24.380387 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.380372    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e81d1990-1c87-4fc4-98f2-db28f4d89b5d" containerName="test-curl-7981-139865525441712"
Sep 07 09:19:24.380474 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.380457    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e81d1990-1c87-4fc4-98f2-db28f4d89b5d" containerName="test-curl-7981-139865525441712"
Sep 07 09:19:24.382665 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.382647    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:19:24.385364 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.385337    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-12fc0fc8\""
Sep 07 09:19:24.385530 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.385411    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-12fc0fc8-dockercfg-htqg6\""
Sep 07 09:19:24.393084 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.393060    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"]
Sep 07 09:19:24.569012 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.568962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8523be35-f03c-45d1-b30e-aa2d0aa0a430-plugins-config-volume\") pod \"payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2\" (UID: \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\") " pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:19:24.569201 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.569098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqrn2\" (UniqueName: \"kubernetes.io/projected/8523be35-f03c-45d1-b30e-aa2d0aa0a430-kube-api-access-wqrn2\") pod \"payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2\" (UID: \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\") " pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:19:24.670575 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.670478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqrn2\" (UniqueName: \"kubernetes.io/projected/8523be35-f03c-45d1-b30e-aa2d0aa0a430-kube-api-access-wqrn2\") pod \"payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2\" (UID: \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\") " pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:19:24.670961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.670933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8523be35-f03c-45d1-b30e-aa2d0aa0a430-plugins-config-volume\") pod \"payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2\" (UID: \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\") " pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:19:24.672265 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.672236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8523be35-f03c-45d1-b30e-aa2d0aa0a430-plugins-config-volume\") pod \"payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2\" (UID: \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\") " pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:19:24.680635 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.680609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqrn2\" (UniqueName: \"kubernetes.io/projected/8523be35-f03c-45d1-b30e-aa2d0aa0a430-kube-api-access-wqrn2\") pod \"payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2\" (UID: \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\") " pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:19:24.696347 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.696316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:19:24.972358 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:24.972333    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"]
Sep 07 09:19:24.973732 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:19:24.973706    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8523be35_f03c_45d1_b30e_aa2d0aa0a430.slice/crio-ecfa5b4135f05d603bca2fc40205b20bd4d062b3c562f7dce635fe1c0b9018ef WatchSource:0}: Error finding container ecfa5b4135f05d603bca2fc40205b20bd4d062b3c562f7dce635fe1c0b9018ef: Status 404 returned error can't find the container with id ecfa5b4135f05d603bca2fc40205b20bd4d062b3c562f7dce635fe1c0b9018ef
Sep 07 09:19:25.613294 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:25.613231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2" event={"ID":"8523be35-f03c-45d1-b30e-aa2d0aa0a430","Type":"ContainerStarted","Data":"d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71"}
Sep 07 09:19:25.613664 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:25.613363    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:19:25.613664 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:25.613390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2" event={"ID":"8523be35-f03c-45d1-b30e-aa2d0aa0a430","Type":"ContainerStarted","Data":"ecfa5b4135f05d603bca2fc40205b20bd4d062b3c562f7dce635fe1c0b9018ef"}
Sep 07 09:19:25.629993 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:25.629939    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2" podStartSLOduration=1.629923447 podStartE2EDuration="1.629923447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:25.629359359 +0000 UTC m=+1223.598529965" watchObservedRunningTime="2026-09-07 09:19:25.629923447 +0000 UTC m=+1223.599094052"
Sep 07 09:19:36.621193 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:19:36.621154    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:21:03.120452 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:03.120416    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"]
Sep 07 09:21:03.120887 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:03.120693    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2" podUID="8523be35-f03c-45d1-b30e-aa2d0aa0a430" containerName="payload-processing" containerID="cri-o://d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71" gracePeriod=30
Sep 07 09:21:03.374356 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:03.374259    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:21:03.453483 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:03.453449    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqrn2\" (UniqueName: \"kubernetes.io/projected/8523be35-f03c-45d1-b30e-aa2d0aa0a430-kube-api-access-wqrn2\") pod \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\" (UID: \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\") "
Sep 07 09:21:03.453680 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:03.453589    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8523be35-f03c-45d1-b30e-aa2d0aa0a430-plugins-config-volume\") pod \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\" (UID: \"8523be35-f03c-45d1-b30e-aa2d0aa0a430\") "
Sep 07 09:21:03.453927 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:03.453902    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8523be35-f03c-45d1-b30e-aa2d0aa0a430-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8523be35-f03c-45d1-b30e-aa2d0aa0a430" (UID: "8523be35-f03c-45d1-b30e-aa2d0aa0a430"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:21:03.455675 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:03.455641    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8523be35-f03c-45d1-b30e-aa2d0aa0a430-kube-api-access-wqrn2" (OuterVolumeSpecName: "kube-api-access-wqrn2") pod "8523be35-f03c-45d1-b30e-aa2d0aa0a430" (UID: "8523be35-f03c-45d1-b30e-aa2d0aa0a430"). InnerVolumeSpecName "kube-api-access-wqrn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:21:03.555215 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:03.555178    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqrn2\" (UniqueName: \"kubernetes.io/projected/8523be35-f03c-45d1-b30e-aa2d0aa0a430-kube-api-access-wqrn2\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:21:03.555215 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:03.555213    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8523be35-f03c-45d1-b30e-aa2d0aa0a430-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:21:04.114961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.114924    2563 generic.go:356] "Generic (PLEG): container finished" podID="8523be35-f03c-45d1-b30e-aa2d0aa0a430" containerID="d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71" exitCode=0
Sep 07 09:21:04.115142 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.114967    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"
Sep 07 09:21:04.115142 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.114992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2" event={"ID":"8523be35-f03c-45d1-b30e-aa2d0aa0a430","Type":"ContainerDied","Data":"d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71"}
Sep 07 09:21:04.115142 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.115035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2" event={"ID":"8523be35-f03c-45d1-b30e-aa2d0aa0a430","Type":"ContainerDied","Data":"ecfa5b4135f05d603bca2fc40205b20bd4d062b3c562f7dce635fe1c0b9018ef"}
Sep 07 09:21:04.115142 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.115057    2563 scope.go:117] "RemoveContainer" containerID="d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71"
Sep 07 09:21:04.124500 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.124337    2563 scope.go:117] "RemoveContainer" containerID="d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71"
Sep 07 09:21:04.124719 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:21:04.124627    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71\": container with ID starting with d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71 not found: ID does not exist" containerID="d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71"
Sep 07 09:21:04.124719 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.124651    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71"} err="failed to get container status \"d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71\": rpc error: code = NotFound desc = could not find container \"d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71\": container with ID starting with d36335fb60e46e96cd334e016857b0d09df56a0d468ce2a55037010747f82b71 not found: ID does not exist"
Sep 07 09:21:04.137118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.137094    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"]
Sep 07 09:21:04.142098 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.142073    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12fc0fc8-6d9896846f-7x5n2"]
Sep 07 09:21:04.465293 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:21:04.465193    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8523be35-f03c-45d1-b30e-aa2d0aa0a430" path="/var/lib/kubelet/pods/8523be35-f03c-45d1-b30e-aa2d0aa0a430/volumes"
Sep 07 09:22:03.190203 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.190169    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7984-a56b92"]
Sep 07 09:22:03.190743 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.190654    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8523be35-f03c-45d1-b30e-aa2d0aa0a430" containerName="payload-processing"
Sep 07 09:22:03.190743 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.190668    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8523be35-f03c-45d1-b30e-aa2d0aa0a430" containerName="payload-processing"
Sep 07 09:22:03.190743 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.190734    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8523be35-f03c-45d1-b30e-aa2d0aa0a430" containerName="payload-processing"
Sep 07 09:22:03.192796 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.192775    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-a56b92"
Sep 07 09:22:03.195151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.195127    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ckkfv\""
Sep 07 09:22:03.199108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.199083    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7984-a56b92"]
Sep 07 09:22:03.244630 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.244593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srbbt\" (UniqueName: \"kubernetes.io/projected/34d34868-5cdb-4448-85cb-d6a0cd96a493-kube-api-access-srbbt\") pod \"mt-curl-7984-a56b92\" (UID: \"34d34868-5cdb-4448-85cb-d6a0cd96a493\") " pod="opendatahub/mt-curl-7984-a56b92"
Sep 07 09:22:03.345382 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.345345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srbbt\" (UniqueName: \"kubernetes.io/projected/34d34868-5cdb-4448-85cb-d6a0cd96a493-kube-api-access-srbbt\") pod \"mt-curl-7984-a56b92\" (UID: \"34d34868-5cdb-4448-85cb-d6a0cd96a493\") " pod="opendatahub/mt-curl-7984-a56b92"
Sep 07 09:22:03.353198 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.353169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srbbt\" (UniqueName: \"kubernetes.io/projected/34d34868-5cdb-4448-85cb-d6a0cd96a493-kube-api-access-srbbt\") pod \"mt-curl-7984-a56b92\" (UID: \"34d34868-5cdb-4448-85cb-d6a0cd96a493\") " pod="opendatahub/mt-curl-7984-a56b92"
Sep 07 09:22:03.503720 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.503688    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-a56b92"
Sep 07 09:22:03.774533 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:03.774505    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7984-a56b92"]
Sep 07 09:22:03.776831 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:22:03.776807    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34d34868_5cdb_4448_85cb_d6a0cd96a493.slice/crio-a60ee9b6394e1f73b4bbf7038f7256b4926f26c3d5571a9c6fef1a2b2a44598e WatchSource:0}: Error finding container a60ee9b6394e1f73b4bbf7038f7256b4926f26c3d5571a9c6fef1a2b2a44598e: Status 404 returned error can't find the container with id a60ee9b6394e1f73b4bbf7038f7256b4926f26c3d5571a9c6fef1a2b2a44598e
Sep 07 09:22:04.423498 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:04.423466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7984-a56b92" event={"ID":"34d34868-5cdb-4448-85cb-d6a0cd96a493","Type":"ContainerStarted","Data":"a60ee9b6394e1f73b4bbf7038f7256b4926f26c3d5571a9c6fef1a2b2a44598e"}
Sep 07 09:22:05.430142 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:05.430107    2563 generic.go:356] "Generic (PLEG): container finished" podID="34d34868-5cdb-4448-85cb-d6a0cd96a493" containerID="2d00a1de66490acc2676a5c8a9ab2bd8777734c9ea5afaf96c7bd2f15cbe61bd" exitCode=0
Sep 07 09:22:05.430548 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:05.430202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7984-a56b92" event={"ID":"34d34868-5cdb-4448-85cb-d6a0cd96a493","Type":"ContainerDied","Data":"2d00a1de66490acc2676a5c8a9ab2bd8777734c9ea5afaf96c7bd2f15cbe61bd"}
Sep 07 09:22:06.576825 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:06.576797    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-a56b92"
Sep 07 09:22:06.608290 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:06.608244    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7984-a56b92_34d34868-5cdb-4448-85cb-d6a0cd96a493/mt-curl-7984-a56b92/0.log"
Sep 07 09:22:06.630403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:06.630363    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7984-a56b92"]
Sep 07 09:22:06.632481 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:06.632458    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7984-a56b92"]
Sep 07 09:22:06.678589 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:06.678555    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srbbt\" (UniqueName: \"kubernetes.io/projected/34d34868-5cdb-4448-85cb-d6a0cd96a493-kube-api-access-srbbt\") pod \"34d34868-5cdb-4448-85cb-d6a0cd96a493\" (UID: \"34d34868-5cdb-4448-85cb-d6a0cd96a493\") "
Sep 07 09:22:06.680638 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:06.680608    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34d34868-5cdb-4448-85cb-d6a0cd96a493-kube-api-access-srbbt" (OuterVolumeSpecName: "kube-api-access-srbbt") pod "34d34868-5cdb-4448-85cb-d6a0cd96a493" (UID: "34d34868-5cdb-4448-85cb-d6a0cd96a493"). InnerVolumeSpecName "kube-api-access-srbbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:22:06.779627 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:06.779583    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srbbt\" (UniqueName: \"kubernetes.io/projected/34d34868-5cdb-4448-85cb-d6a0cd96a493-kube-api-access-srbbt\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:22:07.442698 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:07.442668    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-a56b92"
Sep 07 09:22:07.442888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:07.442720    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a60ee9b6394e1f73b4bbf7038f7256b4926f26c3d5571a9c6fef1a2b2a44598e"
Sep 07 09:22:07.452179 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:22:07.452145    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7984-a56b92\" is forbidden: User \"system:node:ip-10-0-138-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-157.ec2.internal' and this object" podUID="34d34868-5cdb-4448-85cb-d6a0cd96a493" pod="opendatahub/mt-curl-7984-a56b92"
Sep 07 09:22:08.464130 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:08.464098    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34d34868-5cdb-4448-85cb-d6a0cd96a493" path="/var/lib/kubelet/pods/34d34868-5cdb-4448-85cb-d6a0cd96a493/volumes"
Sep 07 09:22:51.900245 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:51.900211    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"]
Sep 07 09:22:51.900843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:51.900823    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34d34868-5cdb-4448-85cb-d6a0cd96a493" containerName="mt-curl-7984-a56b92"
Sep 07 09:22:51.900927 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:51.900846    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34d34868-5cdb-4448-85cb-d6a0cd96a493" containerName="mt-curl-7984-a56b92"
Sep 07 09:22:51.901001 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:51.900988    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34d34868-5cdb-4448-85cb-d6a0cd96a493" containerName="mt-curl-7984-a56b92"
Sep 07 09:22:51.904014 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:51.903989    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:51.906547 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:51.906523    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ce194a-serving-cert\""
Sep 07 09:22:51.913632 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:51.913186    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"]
Sep 07 09:22:52.036765 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.036712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39c7dfe-57f1-445c-a550-8a22a313fcf8-openshift-service-ca\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.036961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.036781    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-maas-api-tls\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.036961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.036818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-metrics-certs\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.036961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.036898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x87vb\" (UniqueName: \"kubernetes.io/projected/b39c7dfe-57f1-445c-a550-8a22a313fcf8-kube-api-access-x87vb\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.137685 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.137637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x87vb\" (UniqueName: \"kubernetes.io/projected/b39c7dfe-57f1-445c-a550-8a22a313fcf8-kube-api-access-x87vb\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.137888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.137743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39c7dfe-57f1-445c-a550-8a22a313fcf8-openshift-service-ca\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.137888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.137782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-maas-api-tls\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.137888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.137812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-metrics-certs\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.138469 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.138440    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39c7dfe-57f1-445c-a550-8a22a313fcf8-openshift-service-ca\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.140503 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.140475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-maas-api-tls\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.140614 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.140483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-metrics-certs\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.152677 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.152613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x87vb\" (UniqueName: \"kubernetes.io/projected/b39c7dfe-57f1-445c-a550-8a22a313fcf8-kube-api-access-x87vb\") pod \"maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.217800 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.217760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:52.489076 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.489051    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"]
Sep 07 09:22:52.491011 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:22:52.490975    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39c7dfe_57f1_445c_a550_8a22a313fcf8.slice/crio-a7a562fd1b5ec8ba047abd652e79b850131c0b7e4d942362b04741a9bd02b356 WatchSource:0}: Error finding container a7a562fd1b5ec8ba047abd652e79b850131c0b7e4d942362b04741a9bd02b356: Status 404 returned error can't find the container with id a7a562fd1b5ec8ba047abd652e79b850131c0b7e4d942362b04741a9bd02b356
Sep 07 09:22:52.678720 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:52.678681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f" event={"ID":"b39c7dfe-57f1-445c-a550-8a22a313fcf8","Type":"ContainerStarted","Data":"a7a562fd1b5ec8ba047abd652e79b850131c0b7e4d942362b04741a9bd02b356"}
Sep 07 09:22:53.686860 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:53.686763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f" event={"ID":"b39c7dfe-57f1-445c-a550-8a22a313fcf8","Type":"ContainerStarted","Data":"285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d"}
Sep 07 09:22:53.687368 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:53.686883    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:22:53.703722 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:53.703662    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f" podStartSLOduration=1.941182365 podStartE2EDuration="2.703641865s" totalImagesPullingTime="762.4595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:22:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:22:53.70170483 +0000 UTC m=+1431.670875435" watchObservedRunningTime="2026-09-07 09:22:53.703641865 +0000 UTC m=+1431.672812470"
Sep 07 09:22:59.697724 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:22:59.697644    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:23:21.828402 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.828357    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"]
Sep 07 09:23:21.832332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.832309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:21.834746 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.834722    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-558343\""
Sep 07 09:23:21.834903 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.834883    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-558343-dockercfg-ncchm\""
Sep 07 09:23:21.844970 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.844943    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"]
Sep 07 09:23:21.877847 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.877816    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"]
Sep 07 09:23:21.881395 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.881375    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:21.891736 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.891711    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"]
Sep 07 09:23:21.936093 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.936058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lcw6\" (UniqueName: \"kubernetes.io/projected/434d1e01-6004-4914-96db-1a2760d1758d-kube-api-access-6lcw6\") pod \"payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8\" (UID: \"434d1e01-6004-4914-96db-1a2760d1758d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:21.936277 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.936120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c5d9109-2d53-4ecb-b2c7-c34759b49695-plugins-config-volume\") pod \"payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8\" (UID: \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\") " pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:21.936277 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.936153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d1e01-6004-4914-96db-1a2760d1758d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8\" (UID: \"434d1e01-6004-4914-96db-1a2760d1758d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:21.936277 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:21.936263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzqtg\" (UniqueName: \"kubernetes.io/projected/0c5d9109-2d53-4ecb-b2c7-c34759b49695-kube-api-access-dzqtg\") pod \"payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8\" (UID: \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\") " pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:22.037462 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.037425    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzqtg\" (UniqueName: \"kubernetes.io/projected/0c5d9109-2d53-4ecb-b2c7-c34759b49695-kube-api-access-dzqtg\") pod \"payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8\" (UID: \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\") " pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:22.037647 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.037588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lcw6\" (UniqueName: \"kubernetes.io/projected/434d1e01-6004-4914-96db-1a2760d1758d-kube-api-access-6lcw6\") pod \"payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8\" (UID: \"434d1e01-6004-4914-96db-1a2760d1758d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:22.037700 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.037649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c5d9109-2d53-4ecb-b2c7-c34759b49695-plugins-config-volume\") pod \"payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8\" (UID: \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\") " pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:22.037700 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.037682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d1e01-6004-4914-96db-1a2760d1758d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8\" (UID: \"434d1e01-6004-4914-96db-1a2760d1758d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:22.038381 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.038359    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c5d9109-2d53-4ecb-b2c7-c34759b49695-plugins-config-volume\") pod \"payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8\" (UID: \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\") " pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:22.038381 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.038378    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d1e01-6004-4914-96db-1a2760d1758d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8\" (UID: \"434d1e01-6004-4914-96db-1a2760d1758d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:22.045673 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.045646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzqtg\" (UniqueName: \"kubernetes.io/projected/0c5d9109-2d53-4ecb-b2c7-c34759b49695-kube-api-access-dzqtg\") pod \"payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8\" (UID: \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\") " pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:22.046186 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.046163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lcw6\" (UniqueName: \"kubernetes.io/projected/434d1e01-6004-4914-96db-1a2760d1758d-kube-api-access-6lcw6\") pod \"payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8\" (UID: \"434d1e01-6004-4914-96db-1a2760d1758d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:22.144301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.144199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:22.191981 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.191944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:22.415997 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.415972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"]
Sep 07 09:23:22.417889 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:23:22.417860    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434d1e01_6004_4914_96db_1a2760d1758d.slice/crio-3cb3d09e1a390aa9945b54a03e293e95c31c7673034ea9cd4773c8adfba55bb8 WatchSource:0}: Error finding container 3cb3d09e1a390aa9945b54a03e293e95c31c7673034ea9cd4773c8adfba55bb8: Status 404 returned error can't find the container with id 3cb3d09e1a390aa9945b54a03e293e95c31c7673034ea9cd4773c8adfba55bb8
Sep 07 09:23:22.479913 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.479888    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"]
Sep 07 09:23:22.481701 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:23:22.481674    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c5d9109_2d53_4ecb_b2c7_c34759b49695.slice/crio-91e1ad0f3eb4936e89f6cc0017398c5ea7540c5dd1c3bfbb2d19afae72fd80f1 WatchSource:0}: Error finding container 91e1ad0f3eb4936e89f6cc0017398c5ea7540c5dd1c3bfbb2d19afae72fd80f1: Status 404 returned error can't find the container with id 91e1ad0f3eb4936e89f6cc0017398c5ea7540c5dd1c3bfbb2d19afae72fd80f1
Sep 07 09:23:22.851787 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.851746    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8" event={"ID":"434d1e01-6004-4914-96db-1a2760d1758d","Type":"ContainerStarted","Data":"c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378"}
Sep 07 09:23:22.852241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.851811    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8" event={"ID":"434d1e01-6004-4914-96db-1a2760d1758d","Type":"ContainerStarted","Data":"3cb3d09e1a390aa9945b54a03e293e95c31c7673034ea9cd4773c8adfba55bb8"}
Sep 07 09:23:22.852241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.851843    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:22.853912 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.853883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8" event={"ID":"0c5d9109-2d53-4ecb-b2c7-c34759b49695","Type":"ContainerStarted","Data":"b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411"}
Sep 07 09:23:22.854040 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.853943    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:22.854040 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.853967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8" event={"ID":"0c5d9109-2d53-4ecb-b2c7-c34759b49695","Type":"ContainerStarted","Data":"91e1ad0f3eb4936e89f6cc0017398c5ea7540c5dd1c3bfbb2d19afae72fd80f1"}
Sep 07 09:23:22.869850 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.869799    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8" podStartSLOduration=1.869782748 podStartE2EDuration="1.869782748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:23:22.86736134 +0000 UTC m=+1460.836531952" watchObservedRunningTime="2026-09-07 09:23:22.869782748 +0000 UTC m=+1460.838953401"
Sep 07 09:23:22.884624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:22.884579    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8" podStartSLOduration=1.8845605380000001 podStartE2EDuration="1.884560538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:23:22.882589318 +0000 UTC m=+1460.851759923" watchObservedRunningTime="2026-09-07 09:23:22.884560538 +0000 UTC m=+1460.853731143"
Sep 07 09:23:33.861969 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:33.861936    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:23:33.862452 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:33.862012    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:23:40.347606 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.347570    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7984-f8bf70"]
Sep 07 09:23:40.351432 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.351410    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-f8bf70"
Sep 07 09:23:40.354374 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.354353    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ckkfv\""
Sep 07 09:23:40.354819 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.354795    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7984-f8bf70"]
Sep 07 09:23:40.522163 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.522120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smfld\" (UniqueName: \"kubernetes.io/projected/3fa0c158-8623-4da3-bad3-be4bb8c6d1e5-kube-api-access-smfld\") pod \"mt-curl-7984-f8bf70\" (UID: \"3fa0c158-8623-4da3-bad3-be4bb8c6d1e5\") " pod="opendatahub/mt-curl-7984-f8bf70"
Sep 07 09:23:40.623614 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.623523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smfld\" (UniqueName: \"kubernetes.io/projected/3fa0c158-8623-4da3-bad3-be4bb8c6d1e5-kube-api-access-smfld\") pod \"mt-curl-7984-f8bf70\" (UID: \"3fa0c158-8623-4da3-bad3-be4bb8c6d1e5\") " pod="opendatahub/mt-curl-7984-f8bf70"
Sep 07 09:23:40.631104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.631071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smfld\" (UniqueName: \"kubernetes.io/projected/3fa0c158-8623-4da3-bad3-be4bb8c6d1e5-kube-api-access-smfld\") pod \"mt-curl-7984-f8bf70\" (UID: \"3fa0c158-8623-4da3-bad3-be4bb8c6d1e5\") " pod="opendatahub/mt-curl-7984-f8bf70"
Sep 07 09:23:40.663656 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.663624    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-f8bf70"
Sep 07 09:23:40.939676 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.939646    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7984-f8bf70"]
Sep 07 09:23:40.941447 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:23:40.941417    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fa0c158_8623_4da3_bad3_be4bb8c6d1e5.slice/crio-2fc8a6bab6f1f3f726f6000dc52843f74ad2a1df4e4dce738bcb32bc1da47166 WatchSource:0}: Error finding container 2fc8a6bab6f1f3f726f6000dc52843f74ad2a1df4e4dce738bcb32bc1da47166: Status 404 returned error can't find the container with id 2fc8a6bab6f1f3f726f6000dc52843f74ad2a1df4e4dce738bcb32bc1da47166
Sep 07 09:23:40.942879 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.942861    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:23:40.961550 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:40.961520    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7984-f8bf70" event={"ID":"3fa0c158-8623-4da3-bad3-be4bb8c6d1e5","Type":"ContainerStarted","Data":"2fc8a6bab6f1f3f726f6000dc52843f74ad2a1df4e4dce738bcb32bc1da47166"}
Sep 07 09:23:41.968352 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:41.968234    2563 generic.go:356] "Generic (PLEG): container finished" podID="3fa0c158-8623-4da3-bad3-be4bb8c6d1e5" containerID="f16a430f43d805ddeb3b973d9cc2e1386694c47e8d3b9808324496df5b24eac4" exitCode=0
Sep 07 09:23:41.968352 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:41.968327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7984-f8bf70" event={"ID":"3fa0c158-8623-4da3-bad3-be4bb8c6d1e5","Type":"ContainerDied","Data":"f16a430f43d805ddeb3b973d9cc2e1386694c47e8d3b9808324496df5b24eac4"}
Sep 07 09:23:43.122692 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.122662    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-f8bf70"
Sep 07 09:23:43.145019 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.144987    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smfld\" (UniqueName: \"kubernetes.io/projected/3fa0c158-8623-4da3-bad3-be4bb8c6d1e5-kube-api-access-smfld\") pod \"3fa0c158-8623-4da3-bad3-be4bb8c6d1e5\" (UID: \"3fa0c158-8623-4da3-bad3-be4bb8c6d1e5\") "
Sep 07 09:23:43.147127 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.147088    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fa0c158-8623-4da3-bad3-be4bb8c6d1e5-kube-api-access-smfld" (OuterVolumeSpecName: "kube-api-access-smfld") pod "3fa0c158-8623-4da3-bad3-be4bb8c6d1e5" (UID: "3fa0c158-8623-4da3-bad3-be4bb8c6d1e5"). InnerVolumeSpecName "kube-api-access-smfld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:23:43.153188 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.153164    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7984-f8bf70_3fa0c158-8623-4da3-bad3-be4bb8c6d1e5/mt-curl-7984-f8bf70/0.log"
Sep 07 09:23:43.173595 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.173566    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7984-f8bf70"]
Sep 07 09:23:43.177108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.177076    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7984-f8bf70"]
Sep 07 09:23:43.246257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.246219    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smfld\" (UniqueName: \"kubernetes.io/projected/3fa0c158-8623-4da3-bad3-be4bb8c6d1e5-kube-api-access-smfld\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:23:43.293630 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.293590    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7984-a429f8"]
Sep 07 09:23:43.294031 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.294017    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fa0c158-8623-4da3-bad3-be4bb8c6d1e5" containerName="mt-curl-7984-f8bf70"
Sep 07 09:23:43.294088 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.294032    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fa0c158-8623-4da3-bad3-be4bb8c6d1e5" containerName="mt-curl-7984-f8bf70"
Sep 07 09:23:43.294132 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.294118    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fa0c158-8623-4da3-bad3-be4bb8c6d1e5" containerName="mt-curl-7984-f8bf70"
Sep 07 09:23:43.297144 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.297124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-a429f8"
Sep 07 09:23:43.303855 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.303828    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7984-a429f8"]
Sep 07 09:23:43.346961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.346921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59mw2\" (UniqueName: \"kubernetes.io/projected/665fe2fd-7322-4f89-8ff9-b636c266a41b-kube-api-access-59mw2\") pod \"mt-curl-7984-a429f8\" (UID: \"665fe2fd-7322-4f89-8ff9-b636c266a41b\") " pod="opendatahub/mt-curl-7984-a429f8"
Sep 07 09:23:43.447867 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.447828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59mw2\" (UniqueName: \"kubernetes.io/projected/665fe2fd-7322-4f89-8ff9-b636c266a41b-kube-api-access-59mw2\") pod \"mt-curl-7984-a429f8\" (UID: \"665fe2fd-7322-4f89-8ff9-b636c266a41b\") " pod="opendatahub/mt-curl-7984-a429f8"
Sep 07 09:23:43.462055 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.462025    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59mw2\" (UniqueName: \"kubernetes.io/projected/665fe2fd-7322-4f89-8ff9-b636c266a41b-kube-api-access-59mw2\") pod \"mt-curl-7984-a429f8\" (UID: \"665fe2fd-7322-4f89-8ff9-b636c266a41b\") " pod="opendatahub/mt-curl-7984-a429f8"
Sep 07 09:23:43.608881 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.608781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-a429f8"
Sep 07 09:23:43.882487 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.882460    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7984-a429f8"]
Sep 07 09:23:43.883692 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:23:43.883662    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod665fe2fd_7322_4f89_8ff9_b636c266a41b.slice/crio-0225d379137a3191c3d3e1b5ed1244b9dccd55784e4ca8482c7745b466a9ffc2 WatchSource:0}: Error finding container 0225d379137a3191c3d3e1b5ed1244b9dccd55784e4ca8482c7745b466a9ffc2: Status 404 returned error can't find the container with id 0225d379137a3191c3d3e1b5ed1244b9dccd55784e4ca8482c7745b466a9ffc2
Sep 07 09:23:43.979853 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.979825    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-f8bf70"
Sep 07 09:23:43.979853 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.979852    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fc8a6bab6f1f3f726f6000dc52843f74ad2a1df4e4dce738bcb32bc1da47166"
Sep 07 09:23:43.981091 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:43.981066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7984-a429f8" event={"ID":"665fe2fd-7322-4f89-8ff9-b636c266a41b","Type":"ContainerStarted","Data":"0225d379137a3191c3d3e1b5ed1244b9dccd55784e4ca8482c7745b466a9ffc2"}
Sep 07 09:23:43.990775 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:23:43.990749    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7984-f8bf70\" is forbidden: User \"system:node:ip-10-0-138-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-157.ec2.internal' and this object" podUID="3fa0c158-8623-4da3-bad3-be4bb8c6d1e5" pod="opendatahub/mt-curl-7984-f8bf70"
Sep 07 09:23:44.464974 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:44.464884    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fa0c158-8623-4da3-bad3-be4bb8c6d1e5" path="/var/lib/kubelet/pods/3fa0c158-8623-4da3-bad3-be4bb8c6d1e5/volumes"
Sep 07 09:23:44.987597 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:44.987562    2563 generic.go:356] "Generic (PLEG): container finished" podID="665fe2fd-7322-4f89-8ff9-b636c266a41b" containerID="0facb4c34f73fda4fe37885fc6b8f0ccb40ef6945fc7084e81b045a2bf3e3a63" exitCode=0
Sep 07 09:23:44.987790 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:44.987643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7984-a429f8" event={"ID":"665fe2fd-7322-4f89-8ff9-b636c266a41b","Type":"ContainerDied","Data":"0facb4c34f73fda4fe37885fc6b8f0ccb40ef6945fc7084e81b045a2bf3e3a63"}
Sep 07 09:23:46.126403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.126374    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-a429f8"
Sep 07 09:23:46.145520 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.145491    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7984-a429f8_665fe2fd-7322-4f89-8ff9-b636c266a41b/mt-curl-7984-a429f8/0.log"
Sep 07 09:23:46.168817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.168779    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7984-a429f8"]
Sep 07 09:23:46.171987 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.171961    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7984-a429f8"]
Sep 07 09:23:46.172933 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.172909    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59mw2\" (UniqueName: \"kubernetes.io/projected/665fe2fd-7322-4f89-8ff9-b636c266a41b-kube-api-access-59mw2\") pod \"665fe2fd-7322-4f89-8ff9-b636c266a41b\" (UID: \"665fe2fd-7322-4f89-8ff9-b636c266a41b\") "
Sep 07 09:23:46.174952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.174923    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/665fe2fd-7322-4f89-8ff9-b636c266a41b-kube-api-access-59mw2" (OuterVolumeSpecName: "kube-api-access-59mw2") pod "665fe2fd-7322-4f89-8ff9-b636c266a41b" (UID: "665fe2fd-7322-4f89-8ff9-b636c266a41b"). InnerVolumeSpecName "kube-api-access-59mw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:23:46.274461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.274429    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59mw2\" (UniqueName: \"kubernetes.io/projected/665fe2fd-7322-4f89-8ff9-b636c266a41b-kube-api-access-59mw2\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:23:46.465929 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.465891    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="665fe2fd-7322-4f89-8ff9-b636c266a41b" path="/var/lib/kubelet/pods/665fe2fd-7322-4f89-8ff9-b636c266a41b/volumes"
Sep 07 09:23:46.999032 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.999003    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7984-a429f8"
Sep 07 09:23:46.999211 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:23:46.999059    2563 scope.go:117] "RemoveContainer" containerID="0facb4c34f73fda4fe37885fc6b8f0ccb40ef6945fc7084e81b045a2bf3e3a63"
Sep 07 09:24:03.397327 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.397273    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"]
Sep 07 09:24:03.397839 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.397579    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f" podUID="b39c7dfe-57f1-445c-a550-8a22a313fcf8" containerName="maas-api" containerID="cri-o://285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d" gracePeriod=30
Sep 07 09:24:03.649158 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.649094    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:24:03.755558 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.755522    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39c7dfe-57f1-445c-a550-8a22a313fcf8-openshift-service-ca\") pod \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") "
Sep 07 09:24:03.755738 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.755581    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-metrics-certs\") pod \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") "
Sep 07 09:24:03.755738 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.755615    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-maas-api-tls\") pod \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") "
Sep 07 09:24:03.755955 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.755728    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x87vb\" (UniqueName: \"kubernetes.io/projected/b39c7dfe-57f1-445c-a550-8a22a313fcf8-kube-api-access-x87vb\") pod \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\" (UID: \"b39c7dfe-57f1-445c-a550-8a22a313fcf8\") "
Sep 07 09:24:03.755955 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.755924    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b39c7dfe-57f1-445c-a550-8a22a313fcf8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b39c7dfe-57f1-445c-a550-8a22a313fcf8" (UID: "b39c7dfe-57f1-445c-a550-8a22a313fcf8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:24:03.756159 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.756141    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39c7dfe-57f1-445c-a550-8a22a313fcf8-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:03.757792 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.757762    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b39c7dfe-57f1-445c-a550-8a22a313fcf8" (UID: "b39c7dfe-57f1-445c-a550-8a22a313fcf8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:24:03.757891 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.757773    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b39c7dfe-57f1-445c-a550-8a22a313fcf8-kube-api-access-x87vb" (OuterVolumeSpecName: "kube-api-access-x87vb") pod "b39c7dfe-57f1-445c-a550-8a22a313fcf8" (UID: "b39c7dfe-57f1-445c-a550-8a22a313fcf8"). InnerVolumeSpecName "kube-api-access-x87vb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:03.757891 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.757793    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b39c7dfe-57f1-445c-a550-8a22a313fcf8" (UID: "b39c7dfe-57f1-445c-a550-8a22a313fcf8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:24:03.857582 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.857544    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:03.857582 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.857576    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39c7dfe-57f1-445c-a550-8a22a313fcf8-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:03.857582 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:03.857587    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x87vb\" (UniqueName: \"kubernetes.io/projected/b39c7dfe-57f1-445c-a550-8a22a313fcf8-kube-api-access-x87vb\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:04.089590 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.089547    2563 generic.go:356] "Generic (PLEG): container finished" podID="b39c7dfe-57f1-445c-a550-8a22a313fcf8" containerID="285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d" exitCode=0
Sep 07 09:24:04.089590 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.089586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f" event={"ID":"b39c7dfe-57f1-445c-a550-8a22a313fcf8","Type":"ContainerDied","Data":"285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d"}
Sep 07 09:24:04.089849 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.089611    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"
Sep 07 09:24:04.089849 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.089645    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f" event={"ID":"b39c7dfe-57f1-445c-a550-8a22a313fcf8","Type":"ContainerDied","Data":"a7a562fd1b5ec8ba047abd652e79b850131c0b7e4d942362b04741a9bd02b356"}
Sep 07 09:24:04.089849 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.089669    2563 scope.go:117] "RemoveContainer" containerID="285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d"
Sep 07 09:24:04.099110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.099089    2563 scope.go:117] "RemoveContainer" containerID="285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d"
Sep 07 09:24:04.099438 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:24:04.099414    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d\": container with ID starting with 285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d not found: ID does not exist" containerID="285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d"
Sep 07 09:24:04.099536 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.099445    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d"} err="failed to get container status \"285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d\": rpc error: code = NotFound desc = could not find container \"285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d\": container with ID starting with 285b61911727a39b245654e290070966d1d4d51b09b86fa3e330d39ff675476d not found: ID does not exist"
Sep 07 09:24:04.111187 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.111164    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"]
Sep 07 09:24:04.112961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.112942    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce194a-54bf8cdb59-phd6f"]
Sep 07 09:24:04.465166 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:04.465074    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b39c7dfe-57f1-445c-a550-8a22a313fcf8" path="/var/lib/kubelet/pods/b39c7dfe-57f1-445c-a550-8a22a313fcf8/volumes"
Sep 07 09:24:25.693242 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:25.693035    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"]
Sep 07 09:24:25.695994 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:25.693316    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8" podUID="0c5d9109-2d53-4ecb-b2c7-c34759b49695" containerName="payload-processing" containerID="cri-o://b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411" gracePeriod=30
Sep 07 09:24:25.700636 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:25.700605    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"]
Sep 07 09:24:25.701196 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:25.700884    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8" podUID="434d1e01-6004-4914-96db-1a2760d1758d" containerName="payload-pre-processing" containerID="cri-o://c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378" gracePeriod=30
Sep 07 09:24:25.993504 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:25.993476    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:24:25.996605 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:25.996579    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:24:26.081848 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.081810    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c5d9109-2d53-4ecb-b2c7-c34759b49695-plugins-config-volume\") pod \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\" (UID: \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\") "
Sep 07 09:24:26.082030 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.081876    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzqtg\" (UniqueName: \"kubernetes.io/projected/0c5d9109-2d53-4ecb-b2c7-c34759b49695-kube-api-access-dzqtg\") pod \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\" (UID: \"0c5d9109-2d53-4ecb-b2c7-c34759b49695\") "
Sep 07 09:24:26.082030 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.081934    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d1e01-6004-4914-96db-1a2760d1758d-plugins-config-volume\") pod \"434d1e01-6004-4914-96db-1a2760d1758d\" (UID: \"434d1e01-6004-4914-96db-1a2760d1758d\") "
Sep 07 09:24:26.082030 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.081972    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lcw6\" (UniqueName: \"kubernetes.io/projected/434d1e01-6004-4914-96db-1a2760d1758d-kube-api-access-6lcw6\") pod \"434d1e01-6004-4914-96db-1a2760d1758d\" (UID: \"434d1e01-6004-4914-96db-1a2760d1758d\") "
Sep 07 09:24:26.082338 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.082312    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/434d1e01-6004-4914-96db-1a2760d1758d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "434d1e01-6004-4914-96db-1a2760d1758d" (UID: "434d1e01-6004-4914-96db-1a2760d1758d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:24:26.082430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.082277    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c5d9109-2d53-4ecb-b2c7-c34759b49695-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c5d9109-2d53-4ecb-b2c7-c34759b49695" (UID: "0c5d9109-2d53-4ecb-b2c7-c34759b49695"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:24:26.083990 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.083968    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c5d9109-2d53-4ecb-b2c7-c34759b49695-kube-api-access-dzqtg" (OuterVolumeSpecName: "kube-api-access-dzqtg") pod "0c5d9109-2d53-4ecb-b2c7-c34759b49695" (UID: "0c5d9109-2d53-4ecb-b2c7-c34759b49695"). InnerVolumeSpecName "kube-api-access-dzqtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:26.084102 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.084079    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434d1e01-6004-4914-96db-1a2760d1758d-kube-api-access-6lcw6" (OuterVolumeSpecName: "kube-api-access-6lcw6") pod "434d1e01-6004-4914-96db-1a2760d1758d" (UID: "434d1e01-6004-4914-96db-1a2760d1758d"). InnerVolumeSpecName "kube-api-access-6lcw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:26.182859 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.182817    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434d1e01-6004-4914-96db-1a2760d1758d-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:26.182859 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.182852    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lcw6\" (UniqueName: \"kubernetes.io/projected/434d1e01-6004-4914-96db-1a2760d1758d-kube-api-access-6lcw6\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:26.183078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.182867    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c5d9109-2d53-4ecb-b2c7-c34759b49695-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:26.183078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.182891    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzqtg\" (UniqueName: \"kubernetes.io/projected/0c5d9109-2d53-4ecb-b2c7-c34759b49695-kube-api-access-dzqtg\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:26.204058 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.204020    2563 generic.go:356] "Generic (PLEG): container finished" podID="0c5d9109-2d53-4ecb-b2c7-c34759b49695" containerID="b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411" exitCode=0
Sep 07 09:24:26.204246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.204073    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"
Sep 07 09:24:26.204246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.204107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8" event={"ID":"0c5d9109-2d53-4ecb-b2c7-c34759b49695","Type":"ContainerDied","Data":"b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411"}
Sep 07 09:24:26.204246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.204162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8" event={"ID":"0c5d9109-2d53-4ecb-b2c7-c34759b49695","Type":"ContainerDied","Data":"91e1ad0f3eb4936e89f6cc0017398c5ea7540c5dd1c3bfbb2d19afae72fd80f1"}
Sep 07 09:24:26.204246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.204188    2563 scope.go:117] "RemoveContainer" containerID="b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411"
Sep 07 09:24:26.205675 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.205649    2563 generic.go:356] "Generic (PLEG): container finished" podID="434d1e01-6004-4914-96db-1a2760d1758d" containerID="c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378" exitCode=0
Sep 07 09:24:26.205776 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.205685    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"
Sep 07 09:24:26.205776 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.205729    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8" event={"ID":"434d1e01-6004-4914-96db-1a2760d1758d","Type":"ContainerDied","Data":"c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378"}
Sep 07 09:24:26.205776 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.205773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8" event={"ID":"434d1e01-6004-4914-96db-1a2760d1758d","Type":"ContainerDied","Data":"3cb3d09e1a390aa9945b54a03e293e95c31c7673034ea9cd4773c8adfba55bb8"}
Sep 07 09:24:26.213395 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.213231    2563 scope.go:117] "RemoveContainer" containerID="b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411"
Sep 07 09:24:26.213682 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:24:26.213661    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411\": container with ID starting with b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411 not found: ID does not exist" containerID="b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411"
Sep 07 09:24:26.213761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.213689    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411"} err="failed to get container status \"b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411\": rpc error: code = NotFound desc = could not find container \"b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411\": container with ID starting with b7fa5d0b27ed4ed085f47204a03d0a538097edafd2890ad13d728fdf3a525411 not found: ID does not exist"
Sep 07 09:24:26.213761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.213707    2563 scope.go:117] "RemoveContainer" containerID="c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378"
Sep 07 09:24:26.221848 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.221829    2563 scope.go:117] "RemoveContainer" containerID="c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378"
Sep 07 09:24:26.222106 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:24:26.222087    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378\": container with ID starting with c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378 not found: ID does not exist" containerID="c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378"
Sep 07 09:24:26.222151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.222113    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378"} err="failed to get container status \"c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378\": rpc error: code = NotFound desc = could not find container \"c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378\": container with ID starting with c596732310eeace500032956334903882ae1587b199459453ca67687cf5a3378 not found: ID does not exist"
Sep 07 09:24:26.226905 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.226883    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"]
Sep 07 09:24:26.230266 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.230248    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-558343-68d6f4c869-4lnw8"]
Sep 07 09:24:26.240236 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.240214    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"]
Sep 07 09:24:26.241944 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.241925    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-558343-7c4dbf44cd-qldd8"]
Sep 07 09:24:26.465043 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.464958    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c5d9109-2d53-4ecb-b2c7-c34759b49695" path="/var/lib/kubelet/pods/0c5d9109-2d53-4ecb-b2c7-c34759b49695/volumes"
Sep 07 09:24:26.465371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:26.465358    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="434d1e01-6004-4914-96db-1a2760d1758d" path="/var/lib/kubelet/pods/434d1e01-6004-4914-96db-1a2760d1758d/volumes"
Sep 07 09:24:42.823953 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:42.823921    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"]
Sep 07 09:24:42.824435 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:42.824170    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6" podUID="5de4ac61-5425-4f69-ab7a-ca8e319579eb" containerName="maas-api" containerID="cri-o://a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0" gracePeriod=30
Sep 07 09:24:42.876891 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:42.876859    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"]
Sep 07 09:24:42.877107 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:42.877076    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j" podUID="ad152bed-9809-4a9b-bc47-67a82cfd0908" containerName="payload-processing" containerID="cri-o://ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601" gracePeriod=30
Sep 07 09:24:42.882250 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:42.882176    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"]
Sep 07 09:24:42.882511 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:42.882413    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv" podUID="3182cfb1-ff92-4a63-9ff5-7b01eb68a48b" containerName="payload-pre-processing" containerID="cri-o://c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2" gracePeriod=30
Sep 07 09:24:43.158395 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.158368    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:24:43.161599 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.161577    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:24:43.168997 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.168970    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:24:43.256587 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.256549    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-plugins-config-volume\") pod \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\" (UID: \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\") "
Sep 07 09:24:43.256763 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.256595    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-metrics-certs\") pod \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") "
Sep 07 09:24:43.256763 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.256622    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de4ac61-5425-4f69-ab7a-ca8e319579eb-openshift-service-ca\") pod \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") "
Sep 07 09:24:43.256763 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.256727    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5dc5\" (UniqueName: \"kubernetes.io/projected/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-kube-api-access-h5dc5\") pod \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\" (UID: \"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b\") "
Sep 07 09:24:43.256906 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.256851    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad152bed-9809-4a9b-bc47-67a82cfd0908-plugins-config-volume\") pod \"ad152bed-9809-4a9b-bc47-67a82cfd0908\" (UID: \"ad152bed-9809-4a9b-bc47-67a82cfd0908\") "
Sep 07 09:24:43.256957 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.256905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbf77\" (UniqueName: \"kubernetes.io/projected/5de4ac61-5425-4f69-ab7a-ca8e319579eb-kube-api-access-jbf77\") pod \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") "
Sep 07 09:24:43.257009 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.256960    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfsrv\" (UniqueName: \"kubernetes.io/projected/ad152bed-9809-4a9b-bc47-67a82cfd0908-kube-api-access-jfsrv\") pod \"ad152bed-9809-4a9b-bc47-67a82cfd0908\" (UID: \"ad152bed-9809-4a9b-bc47-67a82cfd0908\") "
Sep 07 09:24:43.257070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.256956    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5de4ac61-5425-4f69-ab7a-ca8e319579eb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5de4ac61-5425-4f69-ab7a-ca8e319579eb" (UID: "5de4ac61-5425-4f69-ab7a-ca8e319579eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:24:43.257070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.256988    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3182cfb1-ff92-4a63-9ff5-7b01eb68a48b" (UID: "3182cfb1-ff92-4a63-9ff5-7b01eb68a48b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:24:43.257070 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.257004    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-maas-api-tls\") pod \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\" (UID: \"5de4ac61-5425-4f69-ab7a-ca8e319579eb\") "
Sep 07 09:24:43.257330 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.257257    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad152bed-9809-4a9b-bc47-67a82cfd0908-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ad152bed-9809-4a9b-bc47-67a82cfd0908" (UID: "ad152bed-9809-4a9b-bc47-67a82cfd0908"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:24:43.257651 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.257622    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad152bed-9809-4a9b-bc47-67a82cfd0908-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:43.257778 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.257654    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:43.257778 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.257671    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de4ac61-5425-4f69-ab7a-ca8e319579eb-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:43.259011 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.258983    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-kube-api-access-h5dc5" (OuterVolumeSpecName: "kube-api-access-h5dc5") pod "3182cfb1-ff92-4a63-9ff5-7b01eb68a48b" (UID: "3182cfb1-ff92-4a63-9ff5-7b01eb68a48b"). InnerVolumeSpecName "kube-api-access-h5dc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:43.259115 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.259032    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5de4ac61-5425-4f69-ab7a-ca8e319579eb-kube-api-access-jbf77" (OuterVolumeSpecName: "kube-api-access-jbf77") pod "5de4ac61-5425-4f69-ab7a-ca8e319579eb" (UID: "5de4ac61-5425-4f69-ab7a-ca8e319579eb"). InnerVolumeSpecName "kube-api-access-jbf77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:43.259115 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.259049    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5de4ac61-5425-4f69-ab7a-ca8e319579eb" (UID: "5de4ac61-5425-4f69-ab7a-ca8e319579eb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:24:43.259420 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.259404    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5de4ac61-5425-4f69-ab7a-ca8e319579eb" (UID: "5de4ac61-5425-4f69-ab7a-ca8e319579eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:24:43.259475 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.259407    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad152bed-9809-4a9b-bc47-67a82cfd0908-kube-api-access-jfsrv" (OuterVolumeSpecName: "kube-api-access-jfsrv") pod "ad152bed-9809-4a9b-bc47-67a82cfd0908" (UID: "ad152bed-9809-4a9b-bc47-67a82cfd0908"). InnerVolumeSpecName "kube-api-access-jfsrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:43.297471 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.297435    2563 generic.go:356] "Generic (PLEG): container finished" podID="ad152bed-9809-4a9b-bc47-67a82cfd0908" containerID="ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601" exitCode=0
Sep 07 09:24:43.297655 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.297483    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"
Sep 07 09:24:43.297655 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.297518    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j" event={"ID":"ad152bed-9809-4a9b-bc47-67a82cfd0908","Type":"ContainerDied","Data":"ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601"}
Sep 07 09:24:43.297655 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.297590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j" event={"ID":"ad152bed-9809-4a9b-bc47-67a82cfd0908","Type":"ContainerDied","Data":"06c4c10bd681f569da5f2fabab0ca6758c17538c3335e57dcc2d7f8d44f29758"}
Sep 07 09:24:43.297655 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.297626    2563 scope.go:117] "RemoveContainer" containerID="ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601"
Sep 07 09:24:43.298860 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.298833    2563 generic.go:356] "Generic (PLEG): container finished" podID="3182cfb1-ff92-4a63-9ff5-7b01eb68a48b" containerID="c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2" exitCode=0
Sep 07 09:24:43.298933 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.298873    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"
Sep 07 09:24:43.298933 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.298914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv" event={"ID":"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b","Type":"ContainerDied","Data":"c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2"}
Sep 07 09:24:43.299052 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.298963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv" event={"ID":"3182cfb1-ff92-4a63-9ff5-7b01eb68a48b","Type":"ContainerDied","Data":"68e5f003cd5cebcde2b91836d0c906ffb13bf4aa836e0b004989475293464c01"}
Sep 07 09:24:43.300476 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.300437    2563 generic.go:356] "Generic (PLEG): container finished" podID="5de4ac61-5425-4f69-ab7a-ca8e319579eb" containerID="a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0" exitCode=0
Sep 07 09:24:43.300595 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.300486    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"
Sep 07 09:24:43.300595 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.300545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6" event={"ID":"5de4ac61-5425-4f69-ab7a-ca8e319579eb","Type":"ContainerDied","Data":"a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0"}
Sep 07 09:24:43.300701 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.300594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6" event={"ID":"5de4ac61-5425-4f69-ab7a-ca8e319579eb","Type":"ContainerDied","Data":"b4bc52b3bcbc1ae8050381a6cf05c4849b2d72c56377b81cce65a17fd6c22a9f"}
Sep 07 09:24:43.307843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.307824    2563 scope.go:117] "RemoveContainer" containerID="ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601"
Sep 07 09:24:43.308138 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:24:43.308110    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601\": container with ID starting with ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601 not found: ID does not exist" containerID="ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601"
Sep 07 09:24:43.308212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.308151    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601"} err="failed to get container status \"ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601\": rpc error: code = NotFound desc = could not find container \"ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601\": container with ID starting with ab5a4026567a72e2a997e0f01f12c8b2ccc2ee35f48aae7db9332fd184ce5601 not found: ID does not exist"
Sep 07 09:24:43.308212 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.308177    2563 scope.go:117] "RemoveContainer" containerID="c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2"
Sep 07 09:24:43.316495 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.316478    2563 scope.go:117] "RemoveContainer" containerID="c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2"
Sep 07 09:24:43.316769 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:24:43.316752    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2\": container with ID starting with c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2 not found: ID does not exist" containerID="c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2"
Sep 07 09:24:43.316844 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.316779    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2"} err="failed to get container status \"c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2\": rpc error: code = NotFound desc = could not find container \"c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2\": container with ID starting with c2c304039b76b3a16fe999fd94be9440a6d65981d8c9eda5d3ff278819ae9db2 not found: ID does not exist"
Sep 07 09:24:43.316844 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.316801    2563 scope.go:117] "RemoveContainer" containerID="a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0"
Sep 07 09:24:43.325183 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.325160    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"]
Sep 07 09:24:43.325767 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.325752    2563 scope.go:117] "RemoveContainer" containerID="a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0"
Sep 07 09:24:43.326048 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:24:43.326024    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0\": container with ID starting with a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0 not found: ID does not exist" containerID="a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0"
Sep 07 09:24:43.326111 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.326054    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0"} err="failed to get container status \"a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0\": rpc error: code = NotFound desc = could not find container \"a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0\": container with ID starting with a21387bf3b1cbeffe7183ed66cda58446ac560657c84c191a154ec2c0517cec0 not found: ID does not exist"
Sep 07 09:24:43.330104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.330082    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-263421-6fb5b448df-zz86j"]
Sep 07 09:24:43.340292 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.340254    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"]
Sep 07 09:24:43.344965 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.344934    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-263421-86b74bf6b6-55tdv"]
Sep 07 09:24:43.354632 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.354605    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"]
Sep 07 09:24:43.358558 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.358535    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-263421-7b95b84fff-57dk6"]
Sep 07 09:24:43.358960 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.358941    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbf77\" (UniqueName: \"kubernetes.io/projected/5de4ac61-5425-4f69-ab7a-ca8e319579eb-kube-api-access-jbf77\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:43.359005 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.358964    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfsrv\" (UniqueName: \"kubernetes.io/projected/ad152bed-9809-4a9b-bc47-67a82cfd0908-kube-api-access-jfsrv\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:43.359005 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.358978    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:43.359005 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.358986    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de4ac61-5425-4f69-ab7a-ca8e319579eb-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:43.359005 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:43.358994    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5dc5\" (UniqueName: \"kubernetes.io/projected/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b-kube-api-access-h5dc5\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:44.465452 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:44.465419    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3182cfb1-ff92-4a63-9ff5-7b01eb68a48b" path="/var/lib/kubelet/pods/3182cfb1-ff92-4a63-9ff5-7b01eb68a48b/volumes"
Sep 07 09:24:44.465853 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:44.465840    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5de4ac61-5425-4f69-ab7a-ca8e319579eb" path="/var/lib/kubelet/pods/5de4ac61-5425-4f69-ab7a-ca8e319579eb/volumes"
Sep 07 09:24:44.466156 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:44.466144    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad152bed-9809-4a9b-bc47-67a82cfd0908" path="/var/lib/kubelet/pods/ad152bed-9809-4a9b-bc47-67a82cfd0908/volumes"
Sep 07 09:24:49.235383 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:49.235352    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"]
Sep 07 09:24:49.235961 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:49.235565    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy" containerID="cri-o://1e7057a92584d2147833efd32df085f5473ce0501a3e68e068dae5a9abb4bef9" gracePeriod=30
Sep 07 09:24:50.099157 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:50.099114    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 09:24:50.099382 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:50.099177    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 09:24:54.361056 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.361023    2563 generic.go:356] "Generic (PLEG): container finished" podID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerID="1e7057a92584d2147833efd32df085f5473ce0501a3e68e068dae5a9abb4bef9" exitCode=0
Sep 07 09:24:54.361417 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.361090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" event={"ID":"308afb0b-413a-46ce-bbd4-9ef6d000049d","Type":"ContainerDied","Data":"1e7057a92584d2147833efd32df085f5473ce0501a3e68e068dae5a9abb4bef9"}
Sep 07 09:24:54.361417 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.361161    2563 scope.go:117] "RemoveContainer" containerID="8559e36e712cbc561a6aead535392755ff300984770025edff4e71900041f723"
Sep 07 09:24:54.489571 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.489539    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:24:54.568751 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.568640    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-envoy\") pod \"308afb0b-413a-46ce-bbd4-9ef6d000049d\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") "
Sep 07 09:24:54.568751 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.568690    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-credential-socket\") pod \"308afb0b-413a-46ce-bbd4-9ef6d000049d\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") "
Sep 07 09:24:54.569003 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.568839    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308afb0b-413a-46ce-bbd4-9ef6d000049d-istiod-ca-cert\") pod \"308afb0b-413a-46ce-bbd4-9ef6d000049d\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") "
Sep 07 09:24:54.569003 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.568891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f46pp\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-kube-api-access-f46pp\") pod \"308afb0b-413a-46ce-bbd4-9ef6d000049d\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") "
Sep 07 09:24:54.569003 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.568920    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-certs\") pod \"308afb0b-413a-46ce-bbd4-9ef6d000049d\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") "
Sep 07 09:24:54.569003 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.568952    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-token\") pod \"308afb0b-413a-46ce-bbd4-9ef6d000049d\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") "
Sep 07 09:24:54.569003 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.568978    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-data\") pod \"308afb0b-413a-46ce-bbd4-9ef6d000049d\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") "
Sep 07 09:24:54.569268 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569042    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-socket\") pod \"308afb0b-413a-46ce-bbd4-9ef6d000049d\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") "
Sep 07 09:24:54.569268 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569048    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "308afb0b-413a-46ce-bbd4-9ef6d000049d" (UID: "308afb0b-413a-46ce-bbd4-9ef6d000049d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:54.569413 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569386    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "308afb0b-413a-46ce-bbd4-9ef6d000049d" (UID: "308afb0b-413a-46ce-bbd4-9ef6d000049d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:54.569413 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569068    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-podinfo\") pod \"308afb0b-413a-46ce-bbd4-9ef6d000049d\" (UID: \"308afb0b-413a-46ce-bbd4-9ef6d000049d\") "
Sep 07 09:24:54.569519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569185    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "308afb0b-413a-46ce-bbd4-9ef6d000049d" (UID: "308afb0b-413a-46ce-bbd4-9ef6d000049d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:54.569519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569209    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308afb0b-413a-46ce-bbd4-9ef6d000049d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "308afb0b-413a-46ce-bbd4-9ef6d000049d" (UID: "308afb0b-413a-46ce-bbd4-9ef6d000049d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:24:54.569519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569332    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-data" (OuterVolumeSpecName: "istio-data") pod "308afb0b-413a-46ce-bbd4-9ef6d000049d" (UID: "308afb0b-413a-46ce-bbd4-9ef6d000049d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:54.569733 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569715    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308afb0b-413a-46ce-bbd4-9ef6d000049d-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:54.569793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569737    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:54.569793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569751    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:54.569793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569766    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:54.569793 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.569779    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:54.570916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.570891    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "308afb0b-413a-46ce-bbd4-9ef6d000049d" (UID: "308afb0b-413a-46ce-bbd4-9ef6d000049d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:54.571036 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.571017    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "308afb0b-413a-46ce-bbd4-9ef6d000049d" (UID: "308afb0b-413a-46ce-bbd4-9ef6d000049d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:24:54.571428 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.571402    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-token" (OuterVolumeSpecName: "istio-token") pod "308afb0b-413a-46ce-bbd4-9ef6d000049d" (UID: "308afb0b-413a-46ce-bbd4-9ef6d000049d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:54.571518 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.571402    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-kube-api-access-f46pp" (OuterVolumeSpecName: "kube-api-access-f46pp") pod "308afb0b-413a-46ce-bbd4-9ef6d000049d" (UID: "308afb0b-413a-46ce-bbd4-9ef6d000049d"). InnerVolumeSpecName "kube-api-access-f46pp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:54.670962 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.670925    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:54.670962 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.670955    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f46pp\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-kube-api-access-f46pp\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:54.670962 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.670965    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:54.670962 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:54.670974    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308afb0b-413a-46ce-bbd4-9ef6d000049d-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:24:55.367667 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:55.367639    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"
Sep 07 09:24:55.368102 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:55.367636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2" event={"ID":"308afb0b-413a-46ce-bbd4-9ef6d000049d","Type":"ContainerDied","Data":"bcf08b198af3b8ff1f0b1cfb45f110366f069636dfbe1d50dc4054356b099ef9"}
Sep 07 09:24:55.368102 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:55.367772    2563 scope.go:117] "RemoveContainer" containerID="1e7057a92584d2147833efd32df085f5473ce0501a3e68e068dae5a9abb4bef9"
Sep 07 09:24:55.388692 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:55.388665    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"]
Sep 07 09:24:55.392453 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:55.392427    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-263421-openshift-default-75f6dbc846-h4cl2"]
Sep 07 09:24:56.464948 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:24:56.464910    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" path="/var/lib/kubelet/pods/308afb0b-413a-46ce-bbd4-9ef6d000049d/volumes"
Sep 07 09:25:00.089797 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.089764    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"]
Sep 07 09:25:00.090249 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.090022    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl" podUID="0b621e1b-28cd-4a07-af8a-b876ea399907" containerName="payload-processing" containerID="cri-o://9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e" gracePeriod=30
Sep 07 09:25:00.346869 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.346798    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:25:00.396441 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.396403    2563 generic.go:356] "Generic (PLEG): container finished" podID="0b621e1b-28cd-4a07-af8a-b876ea399907" containerID="9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e" exitCode=0
Sep 07 09:25:00.396641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.396445    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"
Sep 07 09:25:00.396641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.396491    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl" event={"ID":"0b621e1b-28cd-4a07-af8a-b876ea399907","Type":"ContainerDied","Data":"9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e"}
Sep 07 09:25:00.396641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.396540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl" event={"ID":"0b621e1b-28cd-4a07-af8a-b876ea399907","Type":"ContainerDied","Data":"a4e98fcb6d3377a4f0fb3612ecc757bd709c6974f2573452039805426e121401"}
Sep 07 09:25:00.396641 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.396568    2563 scope.go:117] "RemoveContainer" containerID="9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e"
Sep 07 09:25:00.405116 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.405099    2563 scope.go:117] "RemoveContainer" containerID="9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e"
Sep 07 09:25:00.405412 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:25:00.405394    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e\": container with ID starting with 9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e not found: ID does not exist" containerID="9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e"
Sep 07 09:25:00.405478 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.405420    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e"} err="failed to get container status \"9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e\": rpc error: code = NotFound desc = could not find container \"9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e\": container with ID starting with 9710db93daa638226acd1b2f7524c7f5c621365b9630a8ada2b8a0bd841d969e not found: ID does not exist"
Sep 07 09:25:00.422778 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.422742    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b621e1b-28cd-4a07-af8a-b876ea399907-plugins-config-volume\") pod \"0b621e1b-28cd-4a07-af8a-b876ea399907\" (UID: \"0b621e1b-28cd-4a07-af8a-b876ea399907\") "
Sep 07 09:25:00.422778 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.422791    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zjpj\" (UniqueName: \"kubernetes.io/projected/0b621e1b-28cd-4a07-af8a-b876ea399907-kube-api-access-9zjpj\") pod \"0b621e1b-28cd-4a07-af8a-b876ea399907\" (UID: \"0b621e1b-28cd-4a07-af8a-b876ea399907\") "
Sep 07 09:25:00.423121 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.423098    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b621e1b-28cd-4a07-af8a-b876ea399907-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b621e1b-28cd-4a07-af8a-b876ea399907" (UID: "0b621e1b-28cd-4a07-af8a-b876ea399907"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:25:00.424827 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.424802    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b621e1b-28cd-4a07-af8a-b876ea399907-kube-api-access-9zjpj" (OuterVolumeSpecName: "kube-api-access-9zjpj") pod "0b621e1b-28cd-4a07-af8a-b876ea399907" (UID: "0b621e1b-28cd-4a07-af8a-b876ea399907"). InnerVolumeSpecName "kube-api-access-9zjpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:25:00.523916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.523868    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b621e1b-28cd-4a07-af8a-b876ea399907-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:25:00.523916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.523909    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zjpj\" (UniqueName: \"kubernetes.io/projected/0b621e1b-28cd-4a07-af8a-b876ea399907-kube-api-access-9zjpj\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:25:00.712236 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.712140    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"]
Sep 07 09:25:00.714155 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:00.714129    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5560ef-5dcd675d9c-k97rl"]
Sep 07 09:25:02.464215 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:02.464180    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b621e1b-28cd-4a07-af8a-b876ea399907" path="/var/lib/kubelet/pods/0b621e1b-28cd-4a07-af8a-b876ea399907/volumes"
Sep 07 09:25:28.375837 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.375804    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"]
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376165    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b621e1b-28cd-4a07-af8a-b876ea399907" containerName="payload-processing"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376178    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b621e1b-28cd-4a07-af8a-b876ea399907" containerName="payload-processing"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376191    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad152bed-9809-4a9b-bc47-67a82cfd0908" containerName="payload-processing"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376197    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad152bed-9809-4a9b-bc47-67a82cfd0908" containerName="payload-processing"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376207    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="665fe2fd-7322-4f89-8ff9-b636c266a41b" containerName="mt-curl-7984-a429f8"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376213    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="665fe2fd-7322-4f89-8ff9-b636c266a41b" containerName="mt-curl-7984-a429f8"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376223    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376228    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376236    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="434d1e01-6004-4914-96db-1a2760d1758d" containerName="payload-pre-processing"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376242    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="434d1e01-6004-4914-96db-1a2760d1758d" containerName="payload-pre-processing"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376250    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5de4ac61-5425-4f69-ab7a-ca8e319579eb" containerName="maas-api"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376255    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5de4ac61-5425-4f69-ab7a-ca8e319579eb" containerName="maas-api"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376260    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376265    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376272    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b39c7dfe-57f1-445c-a550-8a22a313fcf8" containerName="maas-api"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376276    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b39c7dfe-57f1-445c-a550-8a22a313fcf8" containerName="maas-api"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376308    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c5d9109-2d53-4ecb-b2c7-c34759b49695" containerName="payload-processing"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376313    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c5d9109-2d53-4ecb-b2c7-c34759b49695" containerName="payload-processing"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376320    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3182cfb1-ff92-4a63-9ff5-7b01eb68a48b" containerName="payload-pre-processing"
Sep 07 09:25:28.376320 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376324    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3182cfb1-ff92-4a63-9ff5-7b01eb68a48b" containerName="payload-pre-processing"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376379    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b39c7dfe-57f1-445c-a550-8a22a313fcf8" containerName="maas-api"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376387    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376394    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b621e1b-28cd-4a07-af8a-b876ea399907" containerName="payload-processing"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376400    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="434d1e01-6004-4914-96db-1a2760d1758d" containerName="payload-pre-processing"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376405    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="665fe2fd-7322-4f89-8ff9-b636c266a41b" containerName="mt-curl-7984-a429f8"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376412    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3182cfb1-ff92-4a63-9ff5-7b01eb68a48b" containerName="payload-pre-processing"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376418    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c5d9109-2d53-4ecb-b2c7-c34759b49695" containerName="payload-processing"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376423    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5de4ac61-5425-4f69-ab7a-ca8e319579eb" containerName="maas-api"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376428    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad152bed-9809-4a9b-bc47-67a82cfd0908" containerName="payload-processing"
Sep 07 09:25:28.377060 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.376433    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="308afb0b-413a-46ce-bbd4-9ef6d000049d" containerName="istio-proxy"
Sep 07 09:25:28.385008 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.384982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.387547 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.387519    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-0a9a72-openshift-default-dockercfg-r4cpt\""
Sep 07 09:25:28.391214 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.391190    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"]
Sep 07 09:25:28.488111 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.488069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-data\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.488310 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.488143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d478606-5d13-41f1-b315-251583087f65-istiod-ca-cert\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.488310 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.488170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-credential-socket\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.488310 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.488244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d478606-5d13-41f1-b315-251583087f65-istio-podinfo\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.488459 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.488337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-socket\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.488459 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.488363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-certs\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.488459 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.488395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-istio-token\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.488459 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.488431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q2tl\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-kube-api-access-4q2tl\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.488642 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.488462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-envoy\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589201 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d478606-5d13-41f1-b315-251583087f65-istiod-ca-cert\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589201 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-credential-socket\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d478606-5d13-41f1-b315-251583087f65-istio-podinfo\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-socket\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-certs\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-istio-token\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589363    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q2tl\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-kube-api-access-4q2tl\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-envoy\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-data\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589824 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-credential-socket\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589824 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-socket\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589910 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-data\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589959 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589934    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-certs\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.589999 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.589968    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d478606-5d13-41f1-b315-251583087f65-istiod-ca-cert\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.591720 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.591694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-envoy\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.591824 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.591738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d478606-5d13-41f1-b315-251583087f65-istio-podinfo\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.597509 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.597481    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q2tl\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-kube-api-access-4q2tl\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.597618 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.597528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-istio-token\") pod \"e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") " pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.697936 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.697850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:28.975928 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.975902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"]
Sep 07 09:25:28.977962 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:25:28.977934    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d478606_5d13_41f1_b315_251583087f65.slice/crio-f6a2dcf2211d68027fde47711fc4d67697dbd27d48a6005ceb770e64b5a402a3 WatchSource:0}: Error finding container f6a2dcf2211d68027fde47711fc4d67697dbd27d48a6005ceb770e64b5a402a3: Status 404 returned error can't find the container with id f6a2dcf2211d68027fde47711fc4d67697dbd27d48a6005ceb770e64b5a402a3
Sep 07 09:25:28.980241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.980208    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:25:28.980366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.980292    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:25:28.980366 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:28.980323    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:25:29.542033 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:29.541999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" event={"ID":"2d478606-5d13-41f1-b315-251583087f65","Type":"ContainerStarted","Data":"cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770"}
Sep 07 09:25:29.542456 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:29.542053    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" event={"ID":"2d478606-5d13-41f1-b315-251583087f65","Type":"ContainerStarted","Data":"f6a2dcf2211d68027fde47711fc4d67697dbd27d48a6005ceb770e64b5a402a3"}
Sep 07 09:25:29.565315 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:29.565252    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" podStartSLOduration=1.565236197 podStartE2EDuration="1.565236197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:25:29.563015195 +0000 UTC m=+1587.532185800" watchObservedRunningTime="2026-09-07 09:25:29.565236197 +0000 UTC m=+1587.534406797"
Sep 07 09:25:29.698572 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:29.698532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:30.704210 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:30.704175    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:31.551759 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:31.551724    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:31.552665 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:31.552647    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:32.695941 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.695905    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"]
Sep 07 09:25:32.700137 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.700092    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.702504 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.702479    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-0a9a72-serving-cert\""
Sep 07 09:25:32.707982 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.707956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"]
Sep 07 09:25:32.728557 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.728525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.728763 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.728571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2d5t\" (UniqueName: \"kubernetes.io/projected/43c594cb-295e-4f08-8389-74f31fe744b5-kube-api-access-m2d5t\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.728763 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.728672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-metrics-certs\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.728763 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.728702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c594cb-295e-4f08-8389-74f31fe744b5-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.830131 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.830095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-metrics-certs\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.830131 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.830134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c594cb-295e-4f08-8389-74f31fe744b5-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.830423 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.830194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.830423 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.830212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2d5t\" (UniqueName: \"kubernetes.io/projected/43c594cb-295e-4f08-8389-74f31fe744b5-kube-api-access-m2d5t\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.830423 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:25:32.830339    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-serving-cert: secret "maas-api-e2e-ipp-a-0a9a72-serving-cert" not found
Sep 07 09:25:32.830423 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:25:32.830410    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-maas-api-tls podName:43c594cb-295e-4f08-8389-74f31fe744b5 nodeName:}" failed. No retries permitted until 2026-09-07 09:25:33.330385633 +0000 UTC m=+1591.299556216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-maas-api-tls") pod "maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw" (UID: "43c594cb-295e-4f08-8389-74f31fe744b5") : secret "maas-api-e2e-ipp-a-0a9a72-serving-cert" not found
Sep 07 09:25:32.830859 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.830840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c594cb-295e-4f08-8389-74f31fe744b5-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.832598 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.832576    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-metrics-certs\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:32.842945 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:32.842924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2d5t\" (UniqueName: \"kubernetes.io/projected/43c594cb-295e-4f08-8389-74f31fe744b5-kube-api-access-m2d5t\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:33.335633 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:33.335598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:33.338025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:33.337993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:33.615139 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:33.615057    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:33.887455 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:33.887428    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"]
Sep 07 09:25:33.889189 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:25:33.889161    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c594cb_295e_4f08_8389_74f31fe744b5.slice/crio-241bc5cf54418f711b939521aa6fc85024ebba8ebe7553fe33b5c0053e36ec0b WatchSource:0}: Error finding container 241bc5cf54418f711b939521aa6fc85024ebba8ebe7553fe33b5c0053e36ec0b: Status 404 returned error can't find the container with id 241bc5cf54418f711b939521aa6fc85024ebba8ebe7553fe33b5c0053e36ec0b
Sep 07 09:25:34.567519 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:34.567483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw" event={"ID":"43c594cb-295e-4f08-8389-74f31fe744b5","Type":"ContainerStarted","Data":"19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb"}
Sep 07 09:25:34.567677 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:34.567544    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:34.567677 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:34.567567    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw" event={"ID":"43c594cb-295e-4f08-8389-74f31fe744b5","Type":"ContainerStarted","Data":"241bc5cf54418f711b939521aa6fc85024ebba8ebe7553fe33b5c0053e36ec0b"}
Sep 07 09:25:34.587190 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:34.587140    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw" podStartSLOduration=2.031112903 podStartE2EDuration="2.587126341s" totalImagesPullingTime="556.013438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:25:34.584489091 +0000 UTC m=+1592.553659695" watchObservedRunningTime="2026-09-07 09:25:34.587126341 +0000 UTC m=+1592.556296944"
Sep 07 09:25:40.579228 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:40.579196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:25:46.552648 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:46.552609    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 07 09:25:46.553042 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:46.552680    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 07 09:25:46.632234 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:46.632195    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d478606-5d13-41f1-b315-251583087f65" containerID="cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770" exitCode=0
Sep 07 09:25:46.632407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:46.632265    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" event={"ID":"2d478606-5d13-41f1-b315-251583087f65","Type":"ContainerDied","Data":"cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770"}
Sep 07 09:25:46.632713 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:46.632700    2563 scope.go:117] "RemoveContainer" containerID="cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770"
Sep 07 09:25:46.633750 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:46.633721    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:25:46.633843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:46.633790    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:25:46.633843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:46.633820    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:25:47.640214 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:47.640178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" event={"ID":"2d478606-5d13-41f1-b315-251583087f65","Type":"ContainerStarted","Data":"47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba"}
Sep 07 09:25:47.698612 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:47.698573    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:47.698812 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:47.698789    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 07 09:25:47.698868 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:47.698840    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 07 09:25:48.699190 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:48.699150    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 07 09:25:48.699666 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:48.699212    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 07 09:25:49.698374 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:49.698340    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 07 09:25:49.698542 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:49.698421    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 07 09:25:50.698530 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:50.698499    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 07 09:25:50.698875 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:50.698564    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 07 09:25:51.698715 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:51.698677    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 07 09:25:51.699121 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:51.698734    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 07 09:25:52.698785 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:52.698744    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 07 09:25:52.699258 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:52.698838    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 07 09:25:53.702542 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:53.702512    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:53.703007 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:53.702887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:53.703426 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:53.703409    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:25:57.486613 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.486541    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"]
Sep 07 09:25:57.489079 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.489062    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.491635 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.491612    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-da0a32-openshift-default-dockercfg-88pmz\""
Sep 07 09:25:57.501563 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.501541    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"]
Sep 07 09:25:57.580095 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.580052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-credential-socket\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.580095 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.580093    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfjgc\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-kube-api-access-pfjgc\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.580371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.580147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-podinfo\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.580371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.580194    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-socket\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.580371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.580225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-token\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.580371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.580244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-data\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.580371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.580305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-envoy\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.580371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.580329    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/958c61bf-2926-4b56-865a-dd5265b26d3e-istiod-ca-cert\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.580594 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.580407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-certs\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.681463 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-socket\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.681463 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-token\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.681734 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-data\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.681734 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-envoy\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.681734 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/958c61bf-2926-4b56-865a-dd5265b26d3e-istiod-ca-cert\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.681734 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-certs\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.681734 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-credential-socket\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.681734 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfjgc\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-kube-api-access-pfjgc\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.681734 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-podinfo\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.682087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-socket\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.682087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.681962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-credential-socket\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.682087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.682030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-certs\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.682193 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.682106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-data\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.682342 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.682322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/958c61bf-2926-4b56-865a-dd5265b26d3e-istiod-ca-cert\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.684011 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.683986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-envoy\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.684156 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.684136    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-podinfo\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.689430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.689408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-token\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.689569 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.689549    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfjgc\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-kube-api-access-pfjgc\") pod \"e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") " pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:57.800677 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:57.800645    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:58.075613 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:58.075589    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"]
Sep 07 09:25:58.077538 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:25:58.077495    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958c61bf_2926_4b56_865a_dd5265b26d3e.slice/crio-41990dfe31b1529c9fe9dab9aa65430283d13f32a135d63b415e3ccae6623c70 WatchSource:0}: Error finding container 41990dfe31b1529c9fe9dab9aa65430283d13f32a135d63b415e3ccae6623c70: Status 404 returned error can't find the container with id 41990dfe31b1529c9fe9dab9aa65430283d13f32a135d63b415e3ccae6623c70
Sep 07 09:25:58.079733 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:58.079696    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:25:58.079843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:58.079779    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:25:58.079843 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:58.079824    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:25:58.721894 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:58.721861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" event={"ID":"958c61bf-2926-4b56-865a-dd5265b26d3e","Type":"ContainerStarted","Data":"fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093"}
Sep 07 09:25:58.722269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:58.721910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" event={"ID":"958c61bf-2926-4b56-865a-dd5265b26d3e","Type":"ContainerStarted","Data":"41990dfe31b1529c9fe9dab9aa65430283d13f32a135d63b415e3ccae6623c70"}
Sep 07 09:25:58.744368 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:58.744316    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" podStartSLOduration=1.744300318 podStartE2EDuration="1.744300318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:25:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:25:58.742323665 +0000 UTC m=+1616.711494271" watchObservedRunningTime="2026-09-07 09:25:58.744300318 +0000 UTC m=+1616.713470918"
Sep 07 09:25:58.801587 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:58.801551    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:25:59.806548 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:25:59.806516    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:26:00.731679 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:00.731649    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:26:00.732794 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:00.732775    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:26:02.355006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.354975    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"]
Sep 07 09:26:02.357565 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.357547    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:02.360627 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.360608    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-da0a32\""
Sep 07 09:26:02.360722 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.360614    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-da0a32-dockercfg-zhgq2\""
Sep 07 09:26:02.372330 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.372303    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"]
Sep 07 09:26:02.428068 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.428033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb2f2191-060a-40fe-89de-b2ac776ec665-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2\" (UID: \"fb2f2191-060a-40fe-89de-b2ac776ec665\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:02.428217 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.428075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c6mw\" (UniqueName: \"kubernetes.io/projected/fb2f2191-060a-40fe-89de-b2ac776ec665-kube-api-access-8c6mw\") pod \"payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2\" (UID: \"fb2f2191-060a-40fe-89de-b2ac776ec665\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:02.528882 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.528843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb2f2191-060a-40fe-89de-b2ac776ec665-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2\" (UID: \"fb2f2191-060a-40fe-89de-b2ac776ec665\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:02.528882 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.528885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c6mw\" (UniqueName: \"kubernetes.io/projected/fb2f2191-060a-40fe-89de-b2ac776ec665-kube-api-access-8c6mw\") pod \"payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2\" (UID: \"fb2f2191-060a-40fe-89de-b2ac776ec665\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:02.531430 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.531410    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-da0a32\""
Sep 07 09:26:02.536750 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.536730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c6mw\" (UniqueName: \"kubernetes.io/projected/fb2f2191-060a-40fe-89de-b2ac776ec665-kube-api-access-8c6mw\") pod \"payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2\" (UID: \"fb2f2191-060a-40fe-89de-b2ac776ec665\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:02.540224 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.540203    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb2f2191-060a-40fe-89de-b2ac776ec665-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2\" (UID: \"fb2f2191-060a-40fe-89de-b2ac776ec665\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:02.670124 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.670046    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-da0a32-dockercfg-zhgq2\""
Sep 07 09:26:02.678024 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.677996    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:02.947877 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:02.947850    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"]
Sep 07 09:26:02.949454 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:26:02.949431    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2f2191_060a_40fe_89de_b2ac776ec665.slice/crio-d0d297e5d5e671d3aa174897509571b531f2d31460066932f48d71530673c62c WatchSource:0}: Error finding container d0d297e5d5e671d3aa174897509571b531f2d31460066932f48d71530673c62c: Status 404 returned error can't find the container with id d0d297e5d5e671d3aa174897509571b531f2d31460066932f48d71530673c62c
Sep 07 09:26:03.751107 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:03.751073    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2" event={"ID":"fb2f2191-060a-40fe-89de-b2ac776ec665","Type":"ContainerStarted","Data":"1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc"}
Sep 07 09:26:03.751521 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:03.751131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2" event={"ID":"fb2f2191-060a-40fe-89de-b2ac776ec665","Type":"ContainerStarted","Data":"d0d297e5d5e671d3aa174897509571b531f2d31460066932f48d71530673c62c"}
Sep 07 09:26:03.751521 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:03.751185    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:03.766986 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:03.766943    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2" podStartSLOduration=1.766930532 podStartE2EDuration="1.766930532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:26:03.765853823 +0000 UTC m=+1621.735024427" watchObservedRunningTime="2026-09-07 09:26:03.766930532 +0000 UTC m=+1621.736101135"
Sep 07 09:26:05.103531 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:05.103503    2563 scope.go:117] "RemoveContainer" containerID="04065c08f1a9c06a0cf83c5d2f5d4f01c474ddd278bd8a70d433d499ae457b85"
Sep 07 09:26:05.111893 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:05.111866    2563 scope.go:117] "RemoveContainer" containerID="b59324cc807efbefd0b7ae516852f5d951ecd7642cf6051f34e9454b53c3e7bf"
Sep 07 09:26:14.758356 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:14.758314    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:26:16.063132 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:16.062901    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": read tcp 10.134.0.2:60178->10.134.0.82:15021: read: connection reset by peer" start-of-body=
Sep 07 09:26:16.063132 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:16.062985    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": read tcp 10.134.0.2:60178->10.134.0.82:15021: read: connection reset by peer"
Sep 07 09:26:16.818938 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:16.818900    2563 generic.go:356] "Generic (PLEG): container finished" podID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerID="fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093" exitCode=0
Sep 07 09:26:16.819110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:16.818947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" event={"ID":"958c61bf-2926-4b56-865a-dd5265b26d3e","Type":"ContainerDied","Data":"fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093"}
Sep 07 09:26:16.819391 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:16.819375    2563 scope.go:117] "RemoveContainer" containerID="fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093"
Sep 07 09:26:16.820535 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:16.820507    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:26:16.820589 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:16.820573    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:26:16.820630 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:16.820603    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:26:17.826050 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:17.826021    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" event={"ID":"958c61bf-2926-4b56-865a-dd5265b26d3e","Type":"ContainerStarted","Data":"d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5"}
Sep 07 09:26:18.801397 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:18.801356    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:26:18.801634 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:18.801612    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 07 09:26:18.801683 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:18.801664    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 07 09:26:19.801609 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:19.801569    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 07 09:26:19.801995 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:19.801641    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 07 09:26:20.801960 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:20.801921    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 07 09:26:20.802381 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:20.802016    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 07 09:26:21.801831 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:21.801790    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 07 09:26:21.802028 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:21.801861    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 07 09:26:22.992452 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:22.992398    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:26:22.992979 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:22.992945    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:26:22.994339 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:22.994304    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:26:30.642269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:26:30.642235    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-s55z8_6e49a562-3d44-40a0-a184-b90e5cd812dc/payload-processing/0.log"
Sep 07 09:27:08.557271 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:08.557239    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-s55z8_6e49a562-3d44-40a0-a184-b90e5cd812dc/payload-processing/0.log"
Sep 07 09:27:09.205363 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.205322    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"]
Sep 07 09:27:09.208269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.208246    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.210835 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.210817    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-d9283c-openshift-default-dockercfg-btvt9\""
Sep 07 09:27:09.222614 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.222591    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"]
Sep 07 09:27:09.389681 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.389648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-socket\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.389681 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.389679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.389927 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.389719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-data\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.389927 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.389738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-certs\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.389927 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.389809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-podinfo\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.389927 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.389859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-token\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.390073 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.389930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2psv\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-kube-api-access-s2psv\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.390073 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.389980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-credential-socket\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.390073 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.389998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-envoy\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491036 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.490961    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-token\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491036 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2psv\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-kube-api-access-s2psv\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-credential-socket\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-envoy\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-socket\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-data\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491246 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-certs\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491559 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-podinfo\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491559 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-socket\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491778 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491753    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-credential-socket\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.491941 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-data\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.492011 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491841    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-certs\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.492011 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.491869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.493247 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.493210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-envoy\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.493526 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.493509    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-podinfo\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.498664 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.498638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-token\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.498817 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.498801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2psv\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-kube-api-access-s2psv\") pod \"e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") " pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.517926 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.517904    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:09.790203 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.790179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"]
Sep 07 09:27:09.792705 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:27:09.792667    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7cbe5c0_087a_4b82_b32b_44d163d3a780.slice/crio-bb9797eb830ee6279503aa052772d8670b1a32229009fb19ea1d8d3e1a5c02d7 WatchSource:0}: Error finding container bb9797eb830ee6279503aa052772d8670b1a32229009fb19ea1d8d3e1a5c02d7: Status 404 returned error can't find the container with id bb9797eb830ee6279503aa052772d8670b1a32229009fb19ea1d8d3e1a5c02d7
Sep 07 09:27:09.794863 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.794830    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:27:09.794939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.794895    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:27:09.794939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:09.794922    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:27:10.094536 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:10.094448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" event={"ID":"c7cbe5c0-087a-4b82-b32b-44d163d3a780","Type":"ContainerStarted","Data":"05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44"}
Sep 07 09:27:10.094536 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:10.094510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" event={"ID":"c7cbe5c0-087a-4b82-b32b-44d163d3a780","Type":"ContainerStarted","Data":"bb9797eb830ee6279503aa052772d8670b1a32229009fb19ea1d8d3e1a5c02d7"}
Sep 07 09:27:10.115078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:10.115022    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" podStartSLOduration=1.115004179 podStartE2EDuration="1.115004179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:27:10.113815516 +0000 UTC m=+1688.082986125" watchObservedRunningTime="2026-09-07 09:27:10.115004179 +0000 UTC m=+1688.084174783"
Sep 07 09:27:10.518753 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:10.518718    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:10.523851 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:10.523821    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:11.105636 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:11.105606    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:11.106508 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:11.106490    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:13.689985 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.689943    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"]
Sep 07 09:27:13.692660 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.692635    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:13.695143 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.695122    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d9283c\""
Sep 07 09:27:13.705351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.705321    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"]
Sep 07 09:27:13.833722 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.833669    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f21375-556b-4289-b116-3fbcee19fda8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw\" (UID: \"14f21375-556b-4289-b116-3fbcee19fda8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:13.833922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.833855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s2vl\" (UniqueName: \"kubernetes.io/projected/14f21375-556b-4289-b116-3fbcee19fda8-kube-api-access-7s2vl\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw\" (UID: \"14f21375-556b-4289-b116-3fbcee19fda8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:13.935123 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.935074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s2vl\" (UniqueName: \"kubernetes.io/projected/14f21375-556b-4289-b116-3fbcee19fda8-kube-api-access-7s2vl\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw\" (UID: \"14f21375-556b-4289-b116-3fbcee19fda8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:13.935385 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.935145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f21375-556b-4289-b116-3fbcee19fda8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw\" (UID: \"14f21375-556b-4289-b116-3fbcee19fda8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:13.935860 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.935838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f21375-556b-4289-b116-3fbcee19fda8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw\" (UID: \"14f21375-556b-4289-b116-3fbcee19fda8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:13.943025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:13.942954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s2vl\" (UniqueName: \"kubernetes.io/projected/14f21375-556b-4289-b116-3fbcee19fda8-kube-api-access-7s2vl\") pod \"payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw\" (UID: \"14f21375-556b-4289-b116-3fbcee19fda8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:14.003894 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:14.003852    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:14.272968 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:14.272944    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"]
Sep 07 09:27:14.274647 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:27:14.274621    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f21375_556b_4289_b116_3fbcee19fda8.slice/crio-a7197bc5b00138f30b64d3e0eb85255d4a62162fda288312bb9492e39fff555b WatchSource:0}: Error finding container a7197bc5b00138f30b64d3e0eb85255d4a62162fda288312bb9492e39fff555b: Status 404 returned error can't find the container with id a7197bc5b00138f30b64d3e0eb85255d4a62162fda288312bb9492e39fff555b
Sep 07 09:27:15.127323 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:15.127260    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw" event={"ID":"14f21375-556b-4289-b116-3fbcee19fda8","Type":"ContainerStarted","Data":"ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8"}
Sep 07 09:27:15.127736 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:15.127343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw" event={"ID":"14f21375-556b-4289-b116-3fbcee19fda8","Type":"ContainerStarted","Data":"a7197bc5b00138f30b64d3e0eb85255d4a62162fda288312bb9492e39fff555b"}
Sep 07 09:27:15.127736 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:15.127393    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:15.146496 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:15.146442    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw" podStartSLOduration=2.146426112 podStartE2EDuration="2.146426112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:27:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:27:15.144324258 +0000 UTC m=+1693.113494862" watchObservedRunningTime="2026-09-07 09:27:15.146426112 +0000 UTC m=+1693.115596715"
Sep 07 09:27:26.134224 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:26.134144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:31.214370 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:31.214333    2563 generic.go:356] "Generic (PLEG): container finished" podID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerID="05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44" exitCode=0
Sep 07 09:27:31.214826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:31.214404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" event={"ID":"c7cbe5c0-087a-4b82-b32b-44d163d3a780","Type":"ContainerDied","Data":"05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44"}
Sep 07 09:27:31.214894 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:31.214859    2563 scope.go:117] "RemoveContainer" containerID="05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44"
Sep 07 09:27:31.216132 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:31.216099    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:27:31.216241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:31.216187    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:27:31.216241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:31.216234    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 07 09:27:32.221952 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:32.221913    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" event={"ID":"c7cbe5c0-087a-4b82-b32b-44d163d3a780","Type":"ContainerStarted","Data":"d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff"}
Sep 07 09:27:32.518460 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:32.518428    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:32.518634 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:32.518612    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 07 09:27:32.518678 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:32.518659    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 07 09:27:33.519088 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:33.519047    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 07 09:27:33.519518 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:33.519119    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 07 09:27:34.518675 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:34.518635    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 07 09:27:34.518871 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:34.518701    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 07 09:27:35.519182 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:35.519147    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 07 09:27:35.519576 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:35.519202    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 07 09:27:36.518442 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:36.518398    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 07 09:27:36.518635 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:36.518461    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 07 09:27:37.581967 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:37.581937    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:37.582446 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:37.582314    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:37.583528 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:37.583500    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:43.575358 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:43.575316    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"]
Sep 07 09:27:43.575839 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:43.575590    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw" podUID="14f21375-556b-4289-b116-3fbcee19fda8" containerName="payload-pre-processing" containerID="cri-o://ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8" gracePeriod=30
Sep 07 09:27:43.850802 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:43.850776    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:43.868791 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:43.868767    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s2vl\" (UniqueName: \"kubernetes.io/projected/14f21375-556b-4289-b116-3fbcee19fda8-kube-api-access-7s2vl\") pod \"14f21375-556b-4289-b116-3fbcee19fda8\" (UID: \"14f21375-556b-4289-b116-3fbcee19fda8\") "
Sep 07 09:27:43.868955 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:43.868840    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f21375-556b-4289-b116-3fbcee19fda8-plugins-config-volume\") pod \"14f21375-556b-4289-b116-3fbcee19fda8\" (UID: \"14f21375-556b-4289-b116-3fbcee19fda8\") "
Sep 07 09:27:43.869232 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:43.869209    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14f21375-556b-4289-b116-3fbcee19fda8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14f21375-556b-4289-b116-3fbcee19fda8" (UID: "14f21375-556b-4289-b116-3fbcee19fda8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:27:43.870868 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:43.870835    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f21375-556b-4289-b116-3fbcee19fda8-kube-api-access-7s2vl" (OuterVolumeSpecName: "kube-api-access-7s2vl") pod "14f21375-556b-4289-b116-3fbcee19fda8" (UID: "14f21375-556b-4289-b116-3fbcee19fda8"). InnerVolumeSpecName "kube-api-access-7s2vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:27:43.970570 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:43.970529    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s2vl\" (UniqueName: \"kubernetes.io/projected/14f21375-556b-4289-b116-3fbcee19fda8-kube-api-access-7s2vl\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:43.970570 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:43.970563    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f21375-556b-4289-b116-3fbcee19fda8-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:44.288856 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.288818    2563 generic.go:356] "Generic (PLEG): container finished" podID="14f21375-556b-4289-b116-3fbcee19fda8" containerID="ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8" exitCode=0
Sep 07 09:27:44.289025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.288867    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"
Sep 07 09:27:44.289025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.288904    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw" event={"ID":"14f21375-556b-4289-b116-3fbcee19fda8","Type":"ContainerDied","Data":"ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8"}
Sep 07 09:27:44.289025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.288967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw" event={"ID":"14f21375-556b-4289-b116-3fbcee19fda8","Type":"ContainerDied","Data":"a7197bc5b00138f30b64d3e0eb85255d4a62162fda288312bb9492e39fff555b"}
Sep 07 09:27:44.289025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.288994    2563 scope.go:117] "RemoveContainer" containerID="ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8"
Sep 07 09:27:44.298118 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.298102    2563 scope.go:117] "RemoveContainer" containerID="ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8"
Sep 07 09:27:44.298444 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:27:44.298424    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8\": container with ID starting with ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8 not found: ID does not exist" containerID="ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8"
Sep 07 09:27:44.298498 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.298453    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8"} err="failed to get container status \"ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8\": rpc error: code = NotFound desc = could not find container \"ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8\": container with ID starting with ae9075560a7fdd3dd740322702c02d7e6cc0d691c7b346f050bc0b1c252a99e8 not found: ID does not exist"
Sep 07 09:27:44.310653 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.310631    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"]
Sep 07 09:27:44.315783 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.315763    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d9283c-844466cd54-5mtpw"]
Sep 07 09:27:44.464751 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:44.464721    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14f21375-556b-4289-b116-3fbcee19fda8" path="/var/lib/kubelet/pods/14f21375-556b-4289-b116-3fbcee19fda8/volumes"
Sep 07 09:27:49.949233 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:49.949196    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"]
Sep 07 09:27:49.949731 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:49.949494    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy" containerID="cri-o://d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff" gracePeriod=30
Sep 07 09:27:52.585353 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:52.585318    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 09:27:52.585724 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:52.585374    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 09:27:55.197108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.197084    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:55.281561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281528    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-certs\") pod \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") "
Sep 07 09:27:55.281748 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281601    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-token\") pod \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") "
Sep 07 09:27:55.281748 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281628    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-socket\") pod \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") "
Sep 07 09:27:55.281748 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281666    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-credential-socket\") pod \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") "
Sep 07 09:27:55.281748 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281696    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istiod-ca-cert\") pod \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") "
Sep 07 09:27:55.281748 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281731    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-podinfo\") pod \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") "
Sep 07 09:27:55.281748 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281729    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c7cbe5c0-087a-4b82-b32b-44d163d3a780" (UID: "c7cbe5c0-087a-4b82-b32b-44d163d3a780"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:27:55.282062 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281752    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-envoy\") pod \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") "
Sep 07 09:27:55.282062 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2psv\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-kube-api-access-s2psv\") pod \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") "
Sep 07 09:27:55.282062 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281831    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-data\") pod \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\" (UID: \"c7cbe5c0-087a-4b82-b32b-44d163d3a780\") "
Sep 07 09:27:55.282062 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.281936    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c7cbe5c0-087a-4b82-b32b-44d163d3a780" (UID: "c7cbe5c0-087a-4b82-b32b-44d163d3a780"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:27:55.282257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.282183    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c7cbe5c0-087a-4b82-b32b-44d163d3a780" (UID: "c7cbe5c0-087a-4b82-b32b-44d163d3a780"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:27:55.282257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.282186    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-data" (OuterVolumeSpecName: "istio-data") pod "c7cbe5c0-087a-4b82-b32b-44d163d3a780" (UID: "c7cbe5c0-087a-4b82-b32b-44d163d3a780"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:27:55.282257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.282203    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c7cbe5c0-087a-4b82-b32b-44d163d3a780" (UID: "c7cbe5c0-087a-4b82-b32b-44d163d3a780"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:27:55.282450 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.282269    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:55.282450 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.282321    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:55.282450 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.282336    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:55.282450 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.282349    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:55.283881 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.283850    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c7cbe5c0-087a-4b82-b32b-44d163d3a780" (UID: "c7cbe5c0-087a-4b82-b32b-44d163d3a780"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:27:55.283881 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.283869    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c7cbe5c0-087a-4b82-b32b-44d163d3a780" (UID: "c7cbe5c0-087a-4b82-b32b-44d163d3a780"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:27:55.284049 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.283894    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-token" (OuterVolumeSpecName: "istio-token") pod "c7cbe5c0-087a-4b82-b32b-44d163d3a780" (UID: "c7cbe5c0-087a-4b82-b32b-44d163d3a780"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:27:55.284049 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.284021    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-kube-api-access-s2psv" (OuterVolumeSpecName: "kube-api-access-s2psv") pod "c7cbe5c0-087a-4b82-b32b-44d163d3a780" (UID: "c7cbe5c0-087a-4b82-b32b-44d163d3a780"). InnerVolumeSpecName "kube-api-access-s2psv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:27:55.350916 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.350885    2563 generic.go:356] "Generic (PLEG): container finished" podID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerID="d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff" exitCode=0
Sep 07 09:27:55.351038 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.350933    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"
Sep 07 09:27:55.351038 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.350970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" event={"ID":"c7cbe5c0-087a-4b82-b32b-44d163d3a780","Type":"ContainerDied","Data":"d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff"}
Sep 07 09:27:55.351038 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.351024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw" event={"ID":"c7cbe5c0-087a-4b82-b32b-44d163d3a780","Type":"ContainerDied","Data":"bb9797eb830ee6279503aa052772d8670b1a32229009fb19ea1d8d3e1a5c02d7"}
Sep 07 09:27:55.351159 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.351047    2563 scope.go:117] "RemoveContainer" containerID="d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff"
Sep 07 09:27:55.361703 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.361501    2563 scope.go:117] "RemoveContainer" containerID="05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44"
Sep 07 09:27:55.369482 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.369467    2563 scope.go:117] "RemoveContainer" containerID="d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff"
Sep 07 09:27:55.369730 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:27:55.369713    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff\": container with ID starting with d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff not found: ID does not exist" containerID="d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff"
Sep 07 09:27:55.369802 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.369742    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff"} err="failed to get container status \"d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff\": rpc error: code = NotFound desc = could not find container \"d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff\": container with ID starting with d8ca7ea45db0be94f8256df2ac7624b0ebf3ba6ea21f6b03e9fc62517e7b60ff not found: ID does not exist"
Sep 07 09:27:55.369802 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.369767    2563 scope.go:117] "RemoveContainer" containerID="05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44"
Sep 07 09:27:55.370040 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:27:55.370022    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44\": container with ID starting with 05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44 not found: ID does not exist" containerID="05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44"
Sep 07 09:27:55.370079 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.370045    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44"} err="failed to get container status \"05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44\": rpc error: code = NotFound desc = could not find container \"05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44\": container with ID starting with 05773f069b43a4038a854220ba588eac81dca5ca65d6ed5cebf3cd1fd9cfdc44 not found: ID does not exist"
Sep 07 09:27:55.375477 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.375456    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"]
Sep 07 09:27:55.381720 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.381697    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d9283c-openshift-default-6cccfb8b6b-8cpsw"]
Sep 07 09:27:55.383307 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.383289    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:55.383359 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.383310    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:55.383359 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.383319    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:55.383359 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.383328    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cbe5c0-087a-4b82-b32b-44d163d3a780-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:55.383359 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:55.383342    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2psv\" (UniqueName: \"kubernetes.io/projected/c7cbe5c0-087a-4b82-b32b-44d163d3a780-kube-api-access-s2psv\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:27:56.465450 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:27:56.465416    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" path="/var/lib/kubelet/pods/c7cbe5c0-087a-4b82-b32b-44d163d3a780/volumes"
Sep 07 09:28:05.140420 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:05.140389    2563 scope.go:117] "RemoveContainer" containerID="2d00a1de66490acc2676a5c8a9ab2bd8777734c9ea5afaf96c7bd2f15cbe61bd"
Sep 07 09:28:08.788786 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:08.788749    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"]
Sep 07 09:28:08.789250 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:08.789072    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw" podUID="43c594cb-295e-4f08-8389-74f31fe744b5" containerName="maas-api" containerID="cri-o://19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb" gracePeriod=30
Sep 07 09:28:09.037891 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.037861    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:28:09.133907 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.133810    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-maas-api-tls\") pod \"43c594cb-295e-4f08-8389-74f31fe744b5\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") "
Sep 07 09:28:09.133907 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.133878    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2d5t\" (UniqueName: \"kubernetes.io/projected/43c594cb-295e-4f08-8389-74f31fe744b5-kube-api-access-m2d5t\") pod \"43c594cb-295e-4f08-8389-74f31fe744b5\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") "
Sep 07 09:28:09.133907 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.133900    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c594cb-295e-4f08-8389-74f31fe744b5-openshift-service-ca\") pod \"43c594cb-295e-4f08-8389-74f31fe744b5\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") "
Sep 07 09:28:09.134182 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.133955    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-metrics-certs\") pod \"43c594cb-295e-4f08-8389-74f31fe744b5\" (UID: \"43c594cb-295e-4f08-8389-74f31fe744b5\") "
Sep 07 09:28:09.134306 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.134257    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c594cb-295e-4f08-8389-74f31fe744b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43c594cb-295e-4f08-8389-74f31fe744b5" (UID: "43c594cb-295e-4f08-8389-74f31fe744b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:28:09.136133 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.136100    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "43c594cb-295e-4f08-8389-74f31fe744b5" (UID: "43c594cb-295e-4f08-8389-74f31fe744b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:28:09.136248 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.136163    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c594cb-295e-4f08-8389-74f31fe744b5-kube-api-access-m2d5t" (OuterVolumeSpecName: "kube-api-access-m2d5t") pod "43c594cb-295e-4f08-8389-74f31fe744b5" (UID: "43c594cb-295e-4f08-8389-74f31fe744b5"). InnerVolumeSpecName "kube-api-access-m2d5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:28:09.136433 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.136308    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43c594cb-295e-4f08-8389-74f31fe744b5" (UID: "43c594cb-295e-4f08-8389-74f31fe744b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:28:09.235078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.235044    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:28:09.235078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.235076    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2d5t\" (UniqueName: \"kubernetes.io/projected/43c594cb-295e-4f08-8389-74f31fe744b5-kube-api-access-m2d5t\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:28:09.235078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.235088    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c594cb-295e-4f08-8389-74f31fe744b5-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:28:09.235352 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.235097    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c594cb-295e-4f08-8389-74f31fe744b5-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:28:09.429132 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.429041    2563 generic.go:356] "Generic (PLEG): container finished" podID="43c594cb-295e-4f08-8389-74f31fe744b5" containerID="19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb" exitCode=0
Sep 07 09:28:09.429132 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.429091    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"
Sep 07 09:28:09.429132 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.429120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw" event={"ID":"43c594cb-295e-4f08-8389-74f31fe744b5","Type":"ContainerDied","Data":"19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb"}
Sep 07 09:28:09.429447 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.429213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw" event={"ID":"43c594cb-295e-4f08-8389-74f31fe744b5","Type":"ContainerDied","Data":"241bc5cf54418f711b939521aa6fc85024ebba8ebe7553fe33b5c0053e36ec0b"}
Sep 07 09:28:09.429447 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.429237    2563 scope.go:117] "RemoveContainer" containerID="19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb"
Sep 07 09:28:09.439381 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.439354    2563 scope.go:117] "RemoveContainer" containerID="19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb"
Sep 07 09:28:09.439731 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:28:09.439710    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb\": container with ID starting with 19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb not found: ID does not exist" containerID="19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb"
Sep 07 09:28:09.439821 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.439740    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb"} err="failed to get container status \"19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb\": rpc error: code = NotFound desc = could not find container \"19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb\": container with ID starting with 19a1ff54353fdab27d15a9c68560a670aab9cf970d841d7580d25762b28196cb not found: ID does not exist"
Sep 07 09:28:09.451059 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.451021    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"]
Sep 07 09:28:09.454241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:09.454214    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0a9a72-5b596c5ddc-9brrw"]
Sep 07 09:28:10.463936 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:10.463886    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43c594cb-295e-4f08-8389-74f31fe744b5" path="/var/lib/kubelet/pods/43c594cb-295e-4f08-8389-74f31fe744b5/volumes"
Sep 07 09:28:56.812868 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:56.812777    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"]
Sep 07 09:28:56.813459 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:28:56.813050    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy" containerID="cri-o://47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba" gracePeriod=30
Sep 07 09:29:02.065987 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.065953    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:29:02.162264 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162190    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q2tl\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-kube-api-access-4q2tl\") pod \"2d478606-5d13-41f1-b315-251583087f65\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") "
Sep 07 09:29:02.162264 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162231    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d478606-5d13-41f1-b315-251583087f65-istiod-ca-cert\") pod \"2d478606-5d13-41f1-b315-251583087f65\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") "
Sep 07 09:29:02.162264 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162255    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d478606-5d13-41f1-b315-251583087f65-istio-podinfo\") pod \"2d478606-5d13-41f1-b315-251583087f65\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") "
Sep 07 09:29:02.162499 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162383    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-istio-token\") pod \"2d478606-5d13-41f1-b315-251583087f65\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") "
Sep 07 09:29:02.162499 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162438    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-socket\") pod \"2d478606-5d13-41f1-b315-251583087f65\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") "
Sep 07 09:29:02.162499 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162478    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-data\") pod \"2d478606-5d13-41f1-b315-251583087f65\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") "
Sep 07 09:29:02.162660 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162505    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-envoy\") pod \"2d478606-5d13-41f1-b315-251583087f65\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") "
Sep 07 09:29:02.162660 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162564    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-credential-socket\") pod \"2d478606-5d13-41f1-b315-251583087f65\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") "
Sep 07 09:29:02.162660 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162604    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-certs\") pod \"2d478606-5d13-41f1-b315-251583087f65\" (UID: \"2d478606-5d13-41f1-b315-251583087f65\") "
Sep 07 09:29:02.162811 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162660    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d478606-5d13-41f1-b315-251583087f65-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d478606-5d13-41f1-b315-251583087f65" (UID: "2d478606-5d13-41f1-b315-251583087f65"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:29:02.162811 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162753    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d478606-5d13-41f1-b315-251583087f65" (UID: "2d478606-5d13-41f1-b315-251583087f65"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:02.162811 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162743    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d478606-5d13-41f1-b315-251583087f65" (UID: "2d478606-5d13-41f1-b315-251583087f65"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:02.162995 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162963    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d478606-5d13-41f1-b315-251583087f65" (UID: "2d478606-5d13-41f1-b315-251583087f65"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:02.162995 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.162977    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d478606-5d13-41f1-b315-251583087f65" (UID: "2d478606-5d13-41f1-b315-251583087f65"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:02.163108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.163053    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d478606-5d13-41f1-b315-251583087f65-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:02.163108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.163070    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:02.163108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.163084    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:02.163108 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.163097    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:02.164503 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.164476    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d478606-5d13-41f1-b315-251583087f65-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d478606-5d13-41f1-b315-251583087f65" (UID: "2d478606-5d13-41f1-b315-251583087f65"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:29:02.164676 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.164655    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-kube-api-access-4q2tl" (OuterVolumeSpecName: "kube-api-access-4q2tl") pod "2d478606-5d13-41f1-b315-251583087f65" (UID: "2d478606-5d13-41f1-b315-251583087f65"). InnerVolumeSpecName "kube-api-access-4q2tl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:02.164893 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.164873    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d478606-5d13-41f1-b315-251583087f65" (UID: "2d478606-5d13-41f1-b315-251583087f65"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:02.164951 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.164888    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d478606-5d13-41f1-b315-251583087f65" (UID: "2d478606-5d13-41f1-b315-251583087f65"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:02.264031 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.263995    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:02.264031 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.264027    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q2tl\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-kube-api-access-4q2tl\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:02.264031 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.264037    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d478606-5d13-41f1-b315-251583087f65-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:02.264330 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.264046    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d478606-5d13-41f1-b315-251583087f65-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:02.264330 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.264054    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d478606-5d13-41f1-b315-251583087f65-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:02.696760 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.696722    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d478606-5d13-41f1-b315-251583087f65" containerID="47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba" exitCode=0
Sep 07 09:29:02.696970 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.696770    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"
Sep 07 09:29:02.696970 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.696787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" event={"ID":"2d478606-5d13-41f1-b315-251583087f65","Type":"ContainerDied","Data":"47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba"}
Sep 07 09:29:02.696970 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.696870    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl" event={"ID":"2d478606-5d13-41f1-b315-251583087f65","Type":"ContainerDied","Data":"f6a2dcf2211d68027fde47711fc4d67697dbd27d48a6005ceb770e64b5a402a3"}
Sep 07 09:29:02.696970 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.696901    2563 scope.go:117] "RemoveContainer" containerID="47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba"
Sep 07 09:29:02.706178 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.706154    2563 scope.go:117] "RemoveContainer" containerID="cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770"
Sep 07 09:29:02.714467 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.714435    2563 scope.go:117] "RemoveContainer" containerID="47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba"
Sep 07 09:29:02.714772 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:02.714750    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba\": container with ID starting with 47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba not found: ID does not exist" containerID="47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba"
Sep 07 09:29:02.714830 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.714784    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba"} err="failed to get container status \"47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba\": rpc error: code = NotFound desc = could not find container \"47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba\": container with ID starting with 47bb2ffa121e915ce5cec0baad4b3a11f16a677fcdae34f394e8464d4a717aba not found: ID does not exist"
Sep 07 09:29:02.714830 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.714808    2563 scope.go:117] "RemoveContainer" containerID="cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770"
Sep 07 09:29:02.715098 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:02.715076    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770\": container with ID starting with cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770 not found: ID does not exist" containerID="cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770"
Sep 07 09:29:02.715151 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.715105    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770"} err="failed to get container status \"cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770\": rpc error: code = NotFound desc = could not find container \"cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770\": container with ID starting with cccf432fb73c65904d3a2edca1c6f0aef73d2405f940785e81226d275e4c5770 not found: ID does not exist"
Sep 07 09:29:02.715302 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.715263    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"]
Sep 07 09:29:02.718598 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:02.718577    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-0a9a72-openshift-default-7dcb79c6fd-zs7fl"]
Sep 07 09:29:04.464397 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:04.464362    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d478606-5d13-41f1-b315-251583087f65" path="/var/lib/kubelet/pods/2d478606-5d13-41f1-b315-251583087f65/volumes"
Sep 07 09:29:07.702045 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:07.702004    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"]
Sep 07 09:29:07.702724 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:07.702687    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2" podUID="fb2f2191-060a-40fe-89de-b2ac776ec665" containerName="payload-pre-processing" containerID="cri-o://1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc" gracePeriod=30
Sep 07 09:29:07.962679 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:07.962609    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:29:08.024398 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.024362    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb2f2191-060a-40fe-89de-b2ac776ec665-plugins-config-volume\") pod \"fb2f2191-060a-40fe-89de-b2ac776ec665\" (UID: \"fb2f2191-060a-40fe-89de-b2ac776ec665\") "
Sep 07 09:29:08.024603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.024411    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8c6mw\" (UniqueName: \"kubernetes.io/projected/fb2f2191-060a-40fe-89de-b2ac776ec665-kube-api-access-8c6mw\") pod \"fb2f2191-060a-40fe-89de-b2ac776ec665\" (UID: \"fb2f2191-060a-40fe-89de-b2ac776ec665\") "
Sep 07 09:29:08.024804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.024778    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb2f2191-060a-40fe-89de-b2ac776ec665-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fb2f2191-060a-40fe-89de-b2ac776ec665" (UID: "fb2f2191-060a-40fe-89de-b2ac776ec665"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:29:08.026498 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.026478    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb2f2191-060a-40fe-89de-b2ac776ec665-kube-api-access-8c6mw" (OuterVolumeSpecName: "kube-api-access-8c6mw") pod "fb2f2191-060a-40fe-89de-b2ac776ec665" (UID: "fb2f2191-060a-40fe-89de-b2ac776ec665"). InnerVolumeSpecName "kube-api-access-8c6mw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:08.125181 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.125134    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb2f2191-060a-40fe-89de-b2ac776ec665-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:08.125181 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.125178    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8c6mw\" (UniqueName: \"kubernetes.io/projected/fb2f2191-060a-40fe-89de-b2ac776ec665-kube-api-access-8c6mw\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:08.732555 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.732073    2563 generic.go:356] "Generic (PLEG): container finished" podID="fb2f2191-060a-40fe-89de-b2ac776ec665" containerID="1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc" exitCode=0
Sep 07 09:29:08.732555 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.732158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2" event={"ID":"fb2f2191-060a-40fe-89de-b2ac776ec665","Type":"ContainerDied","Data":"1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc"}
Sep 07 09:29:08.732555 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.732175    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"
Sep 07 09:29:08.732555 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.732262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2" event={"ID":"fb2f2191-060a-40fe-89de-b2ac776ec665","Type":"ContainerDied","Data":"d0d297e5d5e671d3aa174897509571b531f2d31460066932f48d71530673c62c"}
Sep 07 09:29:08.732555 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.732324    2563 scope.go:117] "RemoveContainer" containerID="1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc"
Sep 07 09:29:08.747170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.747146    2563 scope.go:117] "RemoveContainer" containerID="1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc"
Sep 07 09:29:08.747744 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:08.747715    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc\": container with ID starting with 1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc not found: ID does not exist" containerID="1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc"
Sep 07 09:29:08.747842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.747751    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc"} err="failed to get container status \"1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc\": rpc error: code = NotFound desc = could not find container \"1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc\": container with ID starting with 1024047a6b4b2d2926cdde1a24c6c812b672c2e8e0bee02551f0d4605eb894cc not found: ID does not exist"
Sep 07 09:29:08.749880 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.749859    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"]
Sep 07 09:29:08.751441 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:08.751423    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-da0a32-69588f7dfd-wv2t2"]
Sep 07 09:29:10.464719 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:10.464685    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb2f2191-060a-40fe-89de-b2ac776ec665" path="/var/lib/kubelet/pods/fb2f2191-060a-40fe-89de-b2ac776ec665/volumes"
Sep 07 09:29:14.471158 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:14.471128    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"]
Sep 07 09:29:14.471636 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:14.471355    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy" containerID="cri-o://d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5" gracePeriod=30
Sep 07 09:29:19.724917 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.724892    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:29:19.796031 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.795988    2563 generic.go:356] "Generic (PLEG): container finished" podID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerID="d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5" exitCode=0
Sep 07 09:29:19.796234 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.796035    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"
Sep 07 09:29:19.796234 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.796067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" event={"ID":"958c61bf-2926-4b56-865a-dd5265b26d3e","Type":"ContainerDied","Data":"d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5"}
Sep 07 09:29:19.796234 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.796128    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s" event={"ID":"958c61bf-2926-4b56-865a-dd5265b26d3e","Type":"ContainerDied","Data":"41990dfe31b1529c9fe9dab9aa65430283d13f32a135d63b415e3ccae6623c70"}
Sep 07 09:29:19.796234 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.796157    2563 scope.go:117] "RemoveContainer" containerID="d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5"
Sep 07 09:29:19.804707 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.804689    2563 scope.go:117] "RemoveContainer" containerID="fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093"
Sep 07 09:29:19.812712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.812685    2563 scope.go:117] "RemoveContainer" containerID="d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5"
Sep 07 09:29:19.813031 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:19.813011    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5\": container with ID starting with d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5 not found: ID does not exist" containerID="d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5"
Sep 07 09:29:19.813078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.813044    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5"} err="failed to get container status \"d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5\": rpc error: code = NotFound desc = could not find container \"d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5\": container with ID starting with d16e765ae4617ebe92c8493126861c30133d4c8b8fd618414814d5016d9e9de5 not found: ID does not exist"
Sep 07 09:29:19.813078 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.813065    2563 scope.go:117] "RemoveContainer" containerID="fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093"
Sep 07 09:29:19.813365 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:19.813342    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093\": container with ID starting with fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093 not found: ID does not exist" containerID="fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093"
Sep 07 09:29:19.813432 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.813375    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093"} err="failed to get container status \"fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093\": rpc error: code = NotFound desc = could not find container \"fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093\": container with ID starting with fe76aa945f812a965c819148ee4abb796c13d48b9510de947ac8072783cd5093 not found: ID does not exist"
Sep 07 09:29:19.841377 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841340    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-credential-socket\") pod \"958c61bf-2926-4b56-865a-dd5265b26d3e\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") "
Sep 07 09:29:19.841577 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841412    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-socket\") pod \"958c61bf-2926-4b56-865a-dd5265b26d3e\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") "
Sep 07 09:29:19.841577 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841466    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfjgc\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-kube-api-access-pfjgc\") pod \"958c61bf-2926-4b56-865a-dd5265b26d3e\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") "
Sep 07 09:29:19.841577 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841492    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-certs\") pod \"958c61bf-2926-4b56-865a-dd5265b26d3e\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") "
Sep 07 09:29:19.841577 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841555    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-podinfo\") pod \"958c61bf-2926-4b56-865a-dd5265b26d3e\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") "
Sep 07 09:29:19.841804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841584    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-envoy\") pod \"958c61bf-2926-4b56-865a-dd5265b26d3e\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") "
Sep 07 09:29:19.841804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841641    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/958c61bf-2926-4b56-865a-dd5265b26d3e-istiod-ca-cert\") pod \"958c61bf-2926-4b56-865a-dd5265b26d3e\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") "
Sep 07 09:29:19.841804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841679    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-data\") pod \"958c61bf-2926-4b56-865a-dd5265b26d3e\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") "
Sep 07 09:29:19.841804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841685    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "958c61bf-2926-4b56-865a-dd5265b26d3e" (UID: "958c61bf-2926-4b56-865a-dd5265b26d3e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:19.841804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841710    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-token\") pod \"958c61bf-2926-4b56-865a-dd5265b26d3e\" (UID: \"958c61bf-2926-4b56-865a-dd5265b26d3e\") "
Sep 07 09:29:19.841804 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841732    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "958c61bf-2926-4b56-865a-dd5265b26d3e" (UID: "958c61bf-2926-4b56-865a-dd5265b26d3e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:19.842084 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841866    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "958c61bf-2926-4b56-865a-dd5265b26d3e" (UID: "958c61bf-2926-4b56-865a-dd5265b26d3e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:19.842084 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.841981    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-data" (OuterVolumeSpecName: "istio-data") pod "958c61bf-2926-4b56-865a-dd5265b26d3e" (UID: "958c61bf-2926-4b56-865a-dd5265b26d3e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:19.842084 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.842010    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958c61bf-2926-4b56-865a-dd5265b26d3e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "958c61bf-2926-4b56-865a-dd5265b26d3e" (UID: "958c61bf-2926-4b56-865a-dd5265b26d3e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:29:19.842084 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.842080    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:19.842230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.842092    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:19.842230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.842107    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:19.842230 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.842119    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:19.843889 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.843846    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "958c61bf-2926-4b56-865a-dd5265b26d3e" (UID: "958c61bf-2926-4b56-865a-dd5265b26d3e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:19.844045 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.843954    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-token" (OuterVolumeSpecName: "istio-token") pod "958c61bf-2926-4b56-865a-dd5265b26d3e" (UID: "958c61bf-2926-4b56-865a-dd5265b26d3e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:19.844184 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.844158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "958c61bf-2926-4b56-865a-dd5265b26d3e" (UID: "958c61bf-2926-4b56-865a-dd5265b26d3e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:29:19.844299 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.844199    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-kube-api-access-pfjgc" (OuterVolumeSpecName: "kube-api-access-pfjgc") pod "958c61bf-2926-4b56-865a-dd5265b26d3e" (UID: "958c61bf-2926-4b56-865a-dd5265b26d3e"). InnerVolumeSpecName "kube-api-access-pfjgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:19.943691 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.943648    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/958c61bf-2926-4b56-865a-dd5265b26d3e-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:19.943691 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.943684    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:19.943691 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.943694    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfjgc\" (UniqueName: \"kubernetes.io/projected/958c61bf-2926-4b56-865a-dd5265b26d3e-kube-api-access-pfjgc\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:19.943691 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.943703    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:19.943971 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:19.943713    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/958c61bf-2926-4b56-865a-dd5265b26d3e-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:20.129873 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:20.129839    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"]
Sep 07 09:29:20.133601 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:20.133570    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-da0a32-openshift-default-58c6cfb886-47c2s"]
Sep 07 09:29:20.464972 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:20.464891    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" path="/var/lib/kubelet/pods/958c61bf-2926-4b56-865a-dd5265b26d3e/volumes"
Sep 07 09:29:31.794207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:31.794170    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"]
Sep 07 09:29:31.794829 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:31.794412    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy" containerID="cri-o://94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669" gracePeriod=30
Sep 07 09:29:36.097896 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:36.097849    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 09:29:36.098380 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:36.097937    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 09:29:37.033091 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.033063    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-podinfo\") pod \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") "
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106134    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-certs\") pod \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") "
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106161    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-socket\") pod \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") "
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106203    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-token\") pod \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") "
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106243    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istiod-ca-cert\") pod \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") "
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106304    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-data\") pod \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") "
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106343    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-envoy\") pod \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") "
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106365    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh6td\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-kube-api-access-wh6td\") pod \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") "
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106400    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-credential-socket\") pod \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\" (UID: \"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0\") "
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106505    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" (UID: "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106622    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" (UID: "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106763    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" (UID: "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106781    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106801    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106778    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" (UID: "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:29:37.108207 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.106554    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" (UID: "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:37.109084 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.108498    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" (UID: "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:37.109084 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.108513    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" (UID: "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:37.109084 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.108524    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" (UID: "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:29:37.109084 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.108622    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-kube-api-access-wh6td" (OuterVolumeSpecName: "kube-api-access-wh6td") pod "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" (UID: "3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0"). InnerVolumeSpecName "kube-api-access-wh6td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:37.207842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.207804    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:37.207842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.207833    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:37.207842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.207843    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh6td\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-kube-api-access-wh6td\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:37.207842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.207852    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:37.208127 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.207861    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:37.208127 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.207869    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:37.208127 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.207876    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:37.887842 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.887803    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerID="94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669" exitCode=0
Sep 07 09:29:37.888036 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.887847    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"
Sep 07 09:29:37.888036 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.887877    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" event={"ID":"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0","Type":"ContainerDied","Data":"94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669"}
Sep 07 09:29:37.888036 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.887955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl" event={"ID":"3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0","Type":"ContainerDied","Data":"af5d208239cfb5fda71cc2ab5fe5a2c016a7827921988de2f4084f115c22bf87"}
Sep 07 09:29:37.888036 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.887977    2563 scope.go:117] "RemoveContainer" containerID="94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669"
Sep 07 09:29:37.896893 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.896877    2563 scope.go:117] "RemoveContainer" containerID="768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f"
Sep 07 09:29:37.909476 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.909451    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"]
Sep 07 09:29:37.909808 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.909643    2563 scope.go:117] "RemoveContainer" containerID="94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669"
Sep 07 09:29:37.909963 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:37.909941    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669\": container with ID starting with 94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669 not found: ID does not exist" containerID="94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669"
Sep 07 09:29:37.910006 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.909980    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669"} err="failed to get container status \"94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669\": rpc error: code = NotFound desc = could not find container \"94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669\": container with ID starting with 94955f2b7a375b0842ba7912c4decd916da6371f8e258d23d501c023f1694669 not found: ID does not exist"
Sep 07 09:29:37.910053 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.910004    2563 scope.go:117] "RemoveContainer" containerID="768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f"
Sep 07 09:29:37.910310 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:37.910274    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f\": container with ID starting with 768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f not found: ID does not exist" containerID="768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f"
Sep 07 09:29:37.910360 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.910319    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f"} err="failed to get container status \"768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f\": rpc error: code = NotFound desc = could not find container \"768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f\": container with ID starting with 768d1dc282d5e105b0088a1703f1261b0a6299136c379e3ed34c2a5a824b124f not found: ID does not exist"
Sep 07 09:29:37.913361 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:37.913339    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3dc031-openshift-default-5788d65768-pmvxl"]
Sep 07 09:29:38.464848 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:38.464816    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" path="/var/lib/kubelet/pods/3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0/volumes"
Sep 07 09:29:42.525434 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.525399    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"]
Sep 07 09:29:42.525854 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.525652    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4" podUID="836c06d2-3ed8-4c0f-b681-18768ffa6221" containerName="payload-processing" containerID="cri-o://c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9" gracePeriod=30
Sep 07 09:29:42.775257 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.775232    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:29:42.860087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.860005    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksf2l\" (UniqueName: \"kubernetes.io/projected/836c06d2-3ed8-4c0f-b681-18768ffa6221-kube-api-access-ksf2l\") pod \"836c06d2-3ed8-4c0f-b681-18768ffa6221\" (UID: \"836c06d2-3ed8-4c0f-b681-18768ffa6221\") "
Sep 07 09:29:42.860087 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.860050    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/836c06d2-3ed8-4c0f-b681-18768ffa6221-plugins-config-volume\") pod \"836c06d2-3ed8-4c0f-b681-18768ffa6221\" (UID: \"836c06d2-3ed8-4c0f-b681-18768ffa6221\") "
Sep 07 09:29:42.860522 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.860484    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/836c06d2-3ed8-4c0f-b681-18768ffa6221-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "836c06d2-3ed8-4c0f-b681-18768ffa6221" (UID: "836c06d2-3ed8-4c0f-b681-18768ffa6221"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:29:42.862216 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.862193    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/836c06d2-3ed8-4c0f-b681-18768ffa6221-kube-api-access-ksf2l" (OuterVolumeSpecName: "kube-api-access-ksf2l") pod "836c06d2-3ed8-4c0f-b681-18768ffa6221" (UID: "836c06d2-3ed8-4c0f-b681-18768ffa6221"). InnerVolumeSpecName "kube-api-access-ksf2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:42.917471 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.917433    2563 generic.go:356] "Generic (PLEG): container finished" podID="836c06d2-3ed8-4c0f-b681-18768ffa6221" containerID="c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9" exitCode=0
Sep 07 09:29:42.917651 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.917480    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"
Sep 07 09:29:42.917651 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.917523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4" event={"ID":"836c06d2-3ed8-4c0f-b681-18768ffa6221","Type":"ContainerDied","Data":"c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9"}
Sep 07 09:29:42.917651 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.917580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4" event={"ID":"836c06d2-3ed8-4c0f-b681-18768ffa6221","Type":"ContainerDied","Data":"b204d2099b01a5484428d0d2be504702449211a29f00ba270e9450637be45407"}
Sep 07 09:29:42.917651 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.917608    2563 scope.go:117] "RemoveContainer" containerID="c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9"
Sep 07 09:29:42.926712 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.926686    2563 scope.go:117] "RemoveContainer" containerID="c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9"
Sep 07 09:29:42.926950 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:42.926928    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9\": container with ID starting with c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9 not found: ID does not exist" containerID="c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9"
Sep 07 09:29:42.927027 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.926958    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9"} err="failed to get container status \"c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9\": rpc error: code = NotFound desc = could not find container \"c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9\": container with ID starting with c89a778d125090112c2832b649256829dcd1840cbbb1347ae50ff85ee1a358d9 not found: ID does not exist"
Sep 07 09:29:42.939126 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.939096    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"]
Sep 07 09:29:42.942024 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.941995    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-34de95-66956cbb4f-nxgb4"]
Sep 07 09:29:42.961495 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.961462    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksf2l\" (UniqueName: \"kubernetes.io/projected/836c06d2-3ed8-4c0f-b681-18768ffa6221-kube-api-access-ksf2l\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:42.961495 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:42.961488    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/836c06d2-3ed8-4c0f-b681-18768ffa6221-plugins-config-volume\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:44.463993 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:44.463954    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="836c06d2-3ed8-4c0f-b681-18768ffa6221" path="/var/lib/kubelet/pods/836c06d2-3ed8-4c0f-b681-18768ffa6221/volumes"
Sep 07 09:29:49.208761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:49.208728    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"]
Sep 07 09:29:49.209248 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:49.208938    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy" containerID="cri-o://1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad" gracePeriod=30
Sep 07 09:29:51.684913 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.684873    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-whncm"]
Sep 07 09:29:51.685328 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.685145    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6957b5f6b-whncm" podUID="32a66938-309e-44da-8c4a-07e1005cf4d6" containerName="manager" containerID="cri-o://50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be" gracePeriod=30
Sep 07 09:29:51.925894 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.925869    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:29:51.962505 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.962417    2563 generic.go:356] "Generic (PLEG): container finished" podID="32a66938-309e-44da-8c4a-07e1005cf4d6" containerID="50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be" exitCode=0
Sep 07 09:29:51.962505 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.962462    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-whncm"
Sep 07 09:29:51.962505 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.962495    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-whncm" event={"ID":"32a66938-309e-44da-8c4a-07e1005cf4d6","Type":"ContainerDied","Data":"50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be"}
Sep 07 09:29:51.962756 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.962549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-whncm" event={"ID":"32a66938-309e-44da-8c4a-07e1005cf4d6","Type":"ContainerDied","Data":"ed1c44ef5675b278f49bbab9d212ec4436c4b69bf439b498cf76c6a1be6b8cb0"}
Sep 07 09:29:51.962756 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.962573    2563 scope.go:117] "RemoveContainer" containerID="50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be"
Sep 07 09:29:51.971395 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.971379    2563 scope.go:117] "RemoveContainer" containerID="50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be"
Sep 07 09:29:51.971683 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:51.971662    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be\": container with ID starting with 50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be not found: ID does not exist" containerID="50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be"
Sep 07 09:29:51.971755 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:51.971690    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be"} err="failed to get container status \"50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be\": rpc error: code = NotFound desc = could not find container \"50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be\": container with ID starting with 50ecc0516a8655f299a0597aba2f37b50dc272fe8f26bcd39d74be6d68a0a4be not found: ID does not exist"
Sep 07 09:29:52.042807 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.042762    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-metrics-certs\") pod \"32a66938-309e-44da-8c4a-07e1005cf4d6\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") "
Sep 07 09:29:52.042807 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.042812    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjsst\" (UniqueName: \"kubernetes.io/projected/32a66938-309e-44da-8c4a-07e1005cf4d6-kube-api-access-rjsst\") pod \"32a66938-309e-44da-8c4a-07e1005cf4d6\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") "
Sep 07 09:29:52.043020 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.042853    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-webhook-cert\") pod \"32a66938-309e-44da-8c4a-07e1005cf4d6\" (UID: \"32a66938-309e-44da-8c4a-07e1005cf4d6\") "
Sep 07 09:29:52.044810 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.044776    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "32a66938-309e-44da-8c4a-07e1005cf4d6" (UID: "32a66938-309e-44da-8c4a-07e1005cf4d6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:29:52.044937 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.044880    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32a66938-309e-44da-8c4a-07e1005cf4d6-kube-api-access-rjsst" (OuterVolumeSpecName: "kube-api-access-rjsst") pod "32a66938-309e-44da-8c4a-07e1005cf4d6" (UID: "32a66938-309e-44da-8c4a-07e1005cf4d6"). InnerVolumeSpecName "kube-api-access-rjsst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:52.044978 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.044934    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "32a66938-309e-44da-8c4a-07e1005cf4d6" (UID: "32a66938-309e-44da-8c4a-07e1005cf4d6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:29:52.144606 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.144551    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:52.144606 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.144599    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjsst\" (UniqueName: \"kubernetes.io/projected/32a66938-309e-44da-8c4a-07e1005cf4d6-kube-api-access-rjsst\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:52.144606 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.144608    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32a66938-309e-44da-8c4a-07e1005cf4d6-webhook-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:52.282737 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.282699    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-whncm"]
Sep 07 09:29:52.286523 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.286499    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-whncm"]
Sep 07 09:29:52.463879 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:52.463838    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32a66938-309e-44da-8c4a-07e1005cf4d6" path="/var/lib/kubelet/pods/32a66938-309e-44da-8c4a-07e1005cf4d6/volumes"
Sep 07 09:29:53.115438 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:53.115402    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 09:29:53.115891 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:53.115464    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 09:29:54.451953 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.451928    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:29:54.566096 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566061    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-podinfo\") pod \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") "
Sep 07 09:29:54.566301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566140    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-socket\") pod \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") "
Sep 07 09:29:54.566301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566158    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hvj4\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-kube-api-access-8hvj4\") pod \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") "
Sep 07 09:29:54.566301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566202    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-envoy\") pod \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") "
Sep 07 09:29:54.566301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566241    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-data\") pod \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") "
Sep 07 09:29:54.566301 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566268    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istiod-ca-cert\") pod \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") "
Sep 07 09:29:54.566561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566323    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-credential-socket\") pod \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") "
Sep 07 09:29:54.566561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566363    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-certs\") pod \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") "
Sep 07 09:29:54.566561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566412    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-token\") pod \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\" (UID: \"950b0191-3908-4fa5-8dc9-82f2a7a1d823\") "
Sep 07 09:29:54.566561 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566507    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "950b0191-3908-4fa5-8dc9-82f2a7a1d823" (UID: "950b0191-3908-4fa5-8dc9-82f2a7a1d823"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:54.567044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566587    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-data" (OuterVolumeSpecName: "istio-data") pod "950b0191-3908-4fa5-8dc9-82f2a7a1d823" (UID: "950b0191-3908-4fa5-8dc9-82f2a7a1d823"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:54.567044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566694    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "950b0191-3908-4fa5-8dc9-82f2a7a1d823" (UID: "950b0191-3908-4fa5-8dc9-82f2a7a1d823"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:54.567044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566735    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "950b0191-3908-4fa5-8dc9-82f2a7a1d823" (UID: "950b0191-3908-4fa5-8dc9-82f2a7a1d823"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:29:54.567044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566822    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:54.567044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566828    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "950b0191-3908-4fa5-8dc9-82f2a7a1d823" (UID: "950b0191-3908-4fa5-8dc9-82f2a7a1d823"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:54.567044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566843    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-data\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:54.567044 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.566858    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-credential-socket\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:54.568474 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.568451    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-kube-api-access-8hvj4" (OuterVolumeSpecName: "kube-api-access-8hvj4") pod "950b0191-3908-4fa5-8dc9-82f2a7a1d823" (UID: "950b0191-3908-4fa5-8dc9-82f2a7a1d823"). InnerVolumeSpecName "kube-api-access-8hvj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:54.568474 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.568463    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-token" (OuterVolumeSpecName: "istio-token") pod "950b0191-3908-4fa5-8dc9-82f2a7a1d823" (UID: "950b0191-3908-4fa5-8dc9-82f2a7a1d823"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:54.568474 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.568453    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "950b0191-3908-4fa5-8dc9-82f2a7a1d823" (UID: "950b0191-3908-4fa5-8dc9-82f2a7a1d823"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 09:29:54.568668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.568621    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "950b0191-3908-4fa5-8dc9-82f2a7a1d823" (UID: "950b0191-3908-4fa5-8dc9-82f2a7a1d823"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:29:54.667980 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.667946    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-envoy\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:54.667980 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.667973    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istiod-ca-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:54.667980 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.667982    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/950b0191-3908-4fa5-8dc9-82f2a7a1d823-workload-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:54.667980 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.667989    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-token\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:54.668278 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.667998    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/950b0191-3908-4fa5-8dc9-82f2a7a1d823-istio-podinfo\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:54.668278 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.668006    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hvj4\" (UniqueName: \"kubernetes.io/projected/950b0191-3908-4fa5-8dc9-82f2a7a1d823-kube-api-access-8hvj4\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:29:54.979135 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.979033    2563 generic.go:356] "Generic (PLEG): container finished" podID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerID="1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad" exitCode=0
Sep 07 09:29:54.979135 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.979089    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"
Sep 07 09:29:54.979135 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.979114    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" event={"ID":"950b0191-3908-4fa5-8dc9-82f2a7a1d823","Type":"ContainerDied","Data":"1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad"}
Sep 07 09:29:54.979454 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.979168    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc" event={"ID":"950b0191-3908-4fa5-8dc9-82f2a7a1d823","Type":"ContainerDied","Data":"5e49d6613144928cb6447a0c3450e0f78394aa04ebd1849197ef5afc8c9d65cf"}
Sep 07 09:29:54.979454 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.979192    2563 scope.go:117] "RemoveContainer" containerID="1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad"
Sep 07 09:29:54.988851 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.988834    2563 scope.go:117] "RemoveContainer" containerID="88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e"
Sep 07 09:29:54.996287 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.996263    2563 scope.go:117] "RemoveContainer" containerID="1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad"
Sep 07 09:29:54.996566 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:54.996546    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad\": container with ID starting with 1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad not found: ID does not exist" containerID="1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad"
Sep 07 09:29:54.996637 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.996574    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad"} err="failed to get container status \"1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad\": rpc error: code = NotFound desc = could not find container \"1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad\": container with ID starting with 1e0b890207c5446dafa83bb394aaa633384a6edc9f76820328136588cbc460ad not found: ID does not exist"
Sep 07 09:29:54.996637 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.996594    2563 scope.go:117] "RemoveContainer" containerID="88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e"
Sep 07 09:29:54.996846 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:29:54.996828    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e\": container with ID starting with 88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e not found: ID does not exist" containerID="88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e"
Sep 07 09:29:54.996886 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:54.996852    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e"} err="failed to get container status \"88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e\": rpc error: code = NotFound desc = could not find container \"88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e\": container with ID starting with 88e5fc044291b3d130f54435a88f34c3c230fbd44ff7cd9abc39cabeb7b2f49e not found: ID does not exist"
Sep 07 09:29:55.000211 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:55.000191    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"]
Sep 07 09:29:55.003696 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:55.003677    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-34de95-openshift-default-7c6cf84f8b-fqcvc"]
Sep 07 09:29:56.464415 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:29:56.464380    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" path="/var/lib/kubelet/pods/950b0191-3908-4fa5-8dc9-82f2a7a1d823/volumes"
Sep 07 09:30:05.183768 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:05.183737    2563 scope.go:117] "RemoveContainer" containerID="f16a430f43d805ddeb3b973d9cc2e1386694c47e8d3b9808324496df5b24eac4"
Sep 07 09:30:53.057858 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.057821    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6957b5f6b-qr95s"]
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058134    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058146    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058157    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058163    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058170    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb2f2191-060a-40fe-89de-b2ac776ec665" containerName="payload-pre-processing"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058176    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb2f2191-060a-40fe-89de-b2ac776ec665" containerName="payload-pre-processing"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058183    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14f21375-556b-4289-b116-3fbcee19fda8" containerName="payload-pre-processing"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058188    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="14f21375-556b-4289-b116-3fbcee19fda8" containerName="payload-pre-processing"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058197    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c594cb-295e-4f08-8389-74f31fe744b5" containerName="maas-api"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058202    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c594cb-295e-4f08-8389-74f31fe744b5" containerName="maas-api"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058208    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058214    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058221    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058227    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058234    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058239    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058248    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a66938-309e-44da-8c4a-07e1005cf4d6" containerName="manager"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058253    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a66938-309e-44da-8c4a-07e1005cf4d6" containerName="manager"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058263    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058268    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058275    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy"
Sep 07 09:30:53.058269 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058297    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058306    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="836c06d2-3ed8-4c0f-b681-18768ffa6221" containerName="payload-processing"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058311    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="836c06d2-3ed8-4c0f-b681-18768ffa6221" containerName="payload-processing"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058316    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058321    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058368    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058376    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058382    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="32a66938-309e-44da-8c4a-07e1005cf4d6" containerName="manager"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058388    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058393    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058397    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058402    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="836c06d2-3ed8-4c0f-b681-18768ffa6221" containerName="payload-processing"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058408    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb2f2191-060a-40fe-89de-b2ac776ec665" containerName="payload-pre-processing"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058413    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e1b4a07-b6b5-47bd-b0e1-dbd79e9f19b0" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058419    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="14f21375-556b-4289-b116-3fbcee19fda8" containerName="payload-pre-processing"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058423    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d478606-5d13-41f1-b315-251583087f65" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058428    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c594cb-295e-4f08-8389-74f31fe744b5" containerName="maas-api"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058433    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy"
Sep 07 09:30:53.058939 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.058437    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy"
Sep 07 09:30:53.061291 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.061266    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.065059 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.065037    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 09:30:53.065211 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.065037    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mk7x4\""
Sep 07 09:30:53.065384 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.065366    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 09:30:53.065453 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.065366    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 09:30:53.071125 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.071099    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-qr95s"]
Sep 07 09:30:53.083896 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.083866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-webhook-cert\") pod \"maas-controller-6957b5f6b-qr95s\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") " pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.084029 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.083909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r8c5\" (UniqueName: \"kubernetes.io/projected/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-kube-api-access-2r8c5\") pod \"maas-controller-6957b5f6b-qr95s\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") " pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.084029 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.083935    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-metrics-certs\") pod \"maas-controller-6957b5f6b-qr95s\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") " pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.184606 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.184569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-webhook-cert\") pod \"maas-controller-6957b5f6b-qr95s\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") " pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.184606 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.184611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8c5\" (UniqueName: \"kubernetes.io/projected/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-kube-api-access-2r8c5\") pod \"maas-controller-6957b5f6b-qr95s\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") " pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.184880 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.184639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-metrics-certs\") pod \"maas-controller-6957b5f6b-qr95s\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") " pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.186968 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.186938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-webhook-cert\") pod \"maas-controller-6957b5f6b-qr95s\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") " pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.186968 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.186962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-metrics-certs\") pod \"maas-controller-6957b5f6b-qr95s\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") " pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.197927 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.197896    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r8c5\" (UniqueName: \"kubernetes.io/projected/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-kube-api-access-2r8c5\") pod \"maas-controller-6957b5f6b-qr95s\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") " pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.372452 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.372365    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:53.643826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.643800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-qr95s"]
Sep 07 09:30:53.645903 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:30:53.645880    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bffad9e_c285_4a3c_90de_2fd5efce6d7f.slice/crio-6c044cb68578d88093b089eef505f53ff74c7c49d5496a99fb75db70174232d9 WatchSource:0}: Error finding container 6c044cb68578d88093b089eef505f53ff74c7c49d5496a99fb75db70174232d9: Status 404 returned error can't find the container with id 6c044cb68578d88093b089eef505f53ff74c7c49d5496a99fb75db70174232d9
Sep 07 09:30:53.647150 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:53.647133    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:30:54.240764 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:54.240725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-qr95s" event={"ID":"9bffad9e-c285-4a3c-90de-2fd5efce6d7f","Type":"ContainerStarted","Data":"6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9"}
Sep 07 09:30:54.241117 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:54.240804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-qr95s" event={"ID":"9bffad9e-c285-4a3c-90de-2fd5efce6d7f","Type":"ContainerStarted","Data":"6c044cb68578d88093b089eef505f53ff74c7c49d5496a99fb75db70174232d9"}
Sep 07 09:30:54.241117 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:54.240889    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:30:54.257733 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:30:54.257687    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6957b5f6b-qr95s" podStartSLOduration=0.847076319 podStartE2EDuration="1.25767539s" totalImagesPullingTime="410.599071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:30:54.256715063 +0000 UTC m=+1912.225885666" watchObservedRunningTime="2026-09-07 09:30:54.25767539 +0000 UTC m=+1912.226846042"
Sep 07 09:31:05.247271 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:05.247240    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:31:14.554429 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.554385    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-qr95s"]
Sep 07 09:31:14.554973 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.554640    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6957b5f6b-qr95s" podUID="9bffad9e-c285-4a3c-90de-2fd5efce6d7f" containerName="manager" containerID="cri-o://6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9" gracePeriod=30
Sep 07 09:31:14.799575 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.799550    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:31:14.872110 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.872004    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-metrics-certs\") pod \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") "
Sep 07 09:31:14.872298 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.872136    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-webhook-cert\") pod \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") "
Sep 07 09:31:14.872298 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.872221    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r8c5\" (UniqueName: \"kubernetes.io/projected/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-kube-api-access-2r8c5\") pod \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\" (UID: \"9bffad9e-c285-4a3c-90de-2fd5efce6d7f\") "
Sep 07 09:31:14.874217 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.874190    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9bffad9e-c285-4a3c-90de-2fd5efce6d7f" (UID: "9bffad9e-c285-4a3c-90de-2fd5efce6d7f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:31:14.874407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.874219    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9bffad9e-c285-4a3c-90de-2fd5efce6d7f" (UID: "9bffad9e-c285-4a3c-90de-2fd5efce6d7f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:31:14.874461 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.874418    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-kube-api-access-2r8c5" (OuterVolumeSpecName: "kube-api-access-2r8c5") pod "9bffad9e-c285-4a3c-90de-2fd5efce6d7f" (UID: "9bffad9e-c285-4a3c-90de-2fd5efce6d7f"). InnerVolumeSpecName "kube-api-access-2r8c5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:31:14.973220 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.973187    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:31:14.973220 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.973215    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-webhook-cert\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:31:14.973220 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:14.973226    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r8c5\" (UniqueName: \"kubernetes.io/projected/9bffad9e-c285-4a3c-90de-2fd5efce6d7f-kube-api-access-2r8c5\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:31:15.337163 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:15.337114    2563 generic.go:356] "Generic (PLEG): container finished" podID="9bffad9e-c285-4a3c-90de-2fd5efce6d7f" containerID="6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9" exitCode=0
Sep 07 09:31:15.337355 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:15.337170    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-qr95s"
Sep 07 09:31:15.337355 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:15.337204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-qr95s" event={"ID":"9bffad9e-c285-4a3c-90de-2fd5efce6d7f","Type":"ContainerDied","Data":"6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9"}
Sep 07 09:31:15.337355 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:15.337257    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-qr95s" event={"ID":"9bffad9e-c285-4a3c-90de-2fd5efce6d7f","Type":"ContainerDied","Data":"6c044cb68578d88093b089eef505f53ff74c7c49d5496a99fb75db70174232d9"}
Sep 07 09:31:15.337355 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:15.337278    2563 scope.go:117] "RemoveContainer" containerID="6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9"
Sep 07 09:31:15.347035 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:15.346952    2563 scope.go:117] "RemoveContainer" containerID="6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9"
Sep 07 09:31:15.348341 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:31:15.348314    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9\": container with ID starting with 6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9 not found: ID does not exist" containerID="6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9"
Sep 07 09:31:15.348341 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:15.348344    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9"} err="failed to get container status \"6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9\": rpc error: code = NotFound desc = could not find container \"6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9\": container with ID starting with 6dc0ad8fb475544a67a4d5daf3390a070ee7bf4b98d3cf11d628603c6f3ddcb9 not found: ID does not exist"
Sep 07 09:31:15.358004 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:15.357982    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-qr95s"]
Sep 07 09:31:15.361831 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:15.361809    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-qr95s"]
Sep 07 09:31:16.120224 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.120184    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6957b5f6b-kv72g"]
Sep 07 09:31:16.120624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.120548    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bffad9e-c285-4a3c-90de-2fd5efce6d7f" containerName="manager"
Sep 07 09:31:16.120624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.120561    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bffad9e-c285-4a3c-90de-2fd5efce6d7f" containerName="manager"
Sep 07 09:31:16.120624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.120579    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy"
Sep 07 09:31:16.120624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.120584    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="958c61bf-2926-4b56-865a-dd5265b26d3e" containerName="istio-proxy"
Sep 07 09:31:16.120624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.120593    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy"
Sep 07 09:31:16.120624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.120598    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="950b0191-3908-4fa5-8dc9-82f2a7a1d823" containerName="istio-proxy"
Sep 07 09:31:16.120822 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.120650    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bffad9e-c285-4a3c-90de-2fd5efce6d7f" containerName="manager"
Sep 07 09:31:16.120822 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.120659    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7cbe5c0-087a-4b82-b32b-44d163d3a780" containerName="istio-proxy"
Sep 07 09:31:16.125025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.125002    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.128412 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.128384    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 09:31:16.128412 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.128395    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 09:31:16.128412 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.128384    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 09:31:16.128608 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.128390    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mk7x4\""
Sep 07 09:31:16.133617 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.133597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-kv72g"]
Sep 07 09:31:16.185639 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.185600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/550e03a0-fe1a-4a1e-9dcc-93a526627f45-metrics-certs\") pod \"maas-controller-6957b5f6b-kv72g\" (UID: \"550e03a0-fe1a-4a1e-9dcc-93a526627f45\") " pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.185792 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.185649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gl5x\" (UniqueName: \"kubernetes.io/projected/550e03a0-fe1a-4a1e-9dcc-93a526627f45-kube-api-access-9gl5x\") pod \"maas-controller-6957b5f6b-kv72g\" (UID: \"550e03a0-fe1a-4a1e-9dcc-93a526627f45\") " pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.185792 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.185669    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/550e03a0-fe1a-4a1e-9dcc-93a526627f45-webhook-cert\") pod \"maas-controller-6957b5f6b-kv72g\" (UID: \"550e03a0-fe1a-4a1e-9dcc-93a526627f45\") " pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.286766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.286719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/550e03a0-fe1a-4a1e-9dcc-93a526627f45-metrics-certs\") pod \"maas-controller-6957b5f6b-kv72g\" (UID: \"550e03a0-fe1a-4a1e-9dcc-93a526627f45\") " pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.286766 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.286772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gl5x\" (UniqueName: \"kubernetes.io/projected/550e03a0-fe1a-4a1e-9dcc-93a526627f45-kube-api-access-9gl5x\") pod \"maas-controller-6957b5f6b-kv72g\" (UID: \"550e03a0-fe1a-4a1e-9dcc-93a526627f45\") " pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.287025 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.286799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/550e03a0-fe1a-4a1e-9dcc-93a526627f45-webhook-cert\") pod \"maas-controller-6957b5f6b-kv72g\" (UID: \"550e03a0-fe1a-4a1e-9dcc-93a526627f45\") " pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.289328 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.289302    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/550e03a0-fe1a-4a1e-9dcc-93a526627f45-webhook-cert\") pod \"maas-controller-6957b5f6b-kv72g\" (UID: \"550e03a0-fe1a-4a1e-9dcc-93a526627f45\") " pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.289457 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.289348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/550e03a0-fe1a-4a1e-9dcc-93a526627f45-metrics-certs\") pod \"maas-controller-6957b5f6b-kv72g\" (UID: \"550e03a0-fe1a-4a1e-9dcc-93a526627f45\") " pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.294085 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.294059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gl5x\" (UniqueName: \"kubernetes.io/projected/550e03a0-fe1a-4a1e-9dcc-93a526627f45-kube-api-access-9gl5x\") pod \"maas-controller-6957b5f6b-kv72g\" (UID: \"550e03a0-fe1a-4a1e-9dcc-93a526627f45\") " pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.435306 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.435182    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:16.463868 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.463826    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bffad9e-c285-4a3c-90de-2fd5efce6d7f" path="/var/lib/kubelet/pods/9bffad9e-c285-4a3c-90de-2fd5efce6d7f/volumes"
Sep 07 09:31:16.705777 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:16.705737    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6957b5f6b-kv72g"]
Sep 07 09:31:16.708226 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:31:16.708202    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod550e03a0_fe1a_4a1e_9dcc_93a526627f45.slice/crio-18c5668731620b852ee6441343250f7a3778adfe7cd793b3cc6956468a0ee13b WatchSource:0}: Error finding container 18c5668731620b852ee6441343250f7a3778adfe7cd793b3cc6956468a0ee13b: Status 404 returned error can't find the container with id 18c5668731620b852ee6441343250f7a3778adfe7cd793b3cc6956468a0ee13b
Sep 07 09:31:17.349831 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:17.349735    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-kv72g" event={"ID":"550e03a0-fe1a-4a1e-9dcc-93a526627f45","Type":"ContainerStarted","Data":"93b15556a8ab1fed0a60e464529e10d9935348af76c3f8e264240d7e3a9e05f8"}
Sep 07 09:31:17.349831 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:17.349794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6957b5f6b-kv72g" event={"ID":"550e03a0-fe1a-4a1e-9dcc-93a526627f45","Type":"ContainerStarted","Data":"18c5668731620b852ee6441343250f7a3778adfe7cd793b3cc6956468a0ee13b"}
Sep 07 09:31:17.350241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:17.349848    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:31:17.366332 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:17.366257    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6957b5f6b-kv72g" podStartSLOduration=0.988849738 podStartE2EDuration="1.366242958s" totalImagesPullingTime="377.39322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:31:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:31:17.36501372 +0000 UTC m=+1935.334184337" watchObservedRunningTime="2026-09-07 09:31:17.366242958 +0000 UTC m=+1935.335413560"
Sep 07 09:31:28.357005 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:31:28.356970    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6957b5f6b-kv72g"
Sep 07 09:33:21.923099 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:21.923058    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"]
Sep 07 09:33:21.926644 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:21.926624    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:33:21.929271 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:21.929249    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-s4rqb\""
Sep 07 09:33:21.938173 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:21.938153    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"]
Sep 07 09:33:21.988237 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:21.988205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/59841c33-ac6e-47df-b17c-ef86c59c08ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rmgzt\" (UID: \"59841c33-ac6e-47df-b17c-ef86c59c08ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:33:21.988424 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:21.988261    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdv7f\" (UniqueName: \"kubernetes.io/projected/59841c33-ac6e-47df-b17c-ef86c59c08ad-kube-api-access-fdv7f\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rmgzt\" (UID: \"59841c33-ac6e-47df-b17c-ef86c59c08ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:33:22.089259 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.089220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/59841c33-ac6e-47df-b17c-ef86c59c08ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rmgzt\" (UID: \"59841c33-ac6e-47df-b17c-ef86c59c08ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:33:22.089428 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.089270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdv7f\" (UniqueName: \"kubernetes.io/projected/59841c33-ac6e-47df-b17c-ef86c59c08ad-kube-api-access-fdv7f\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rmgzt\" (UID: \"59841c33-ac6e-47df-b17c-ef86c59c08ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:33:22.089617 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.089594    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/59841c33-ac6e-47df-b17c-ef86c59c08ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rmgzt\" (UID: \"59841c33-ac6e-47df-b17c-ef86c59c08ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:33:22.098624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.098599    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdv7f\" (UniqueName: \"kubernetes.io/projected/59841c33-ac6e-47df-b17c-ef86c59c08ad-kube-api-access-fdv7f\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-rmgzt\" (UID: \"59841c33-ac6e-47df-b17c-ef86c59c08ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:33:22.236159 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.236061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:33:22.511853 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.511822    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"]
Sep 07 09:33:22.512965 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:33:22.512940    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59841c33_ac6e_47df_b17c_ef86c59c08ad.slice/crio-92e1d5c2c3efdc155ba34a14af8bd8b51e79120da1e0c57b8d4ae96ce37a206d WatchSource:0}: Error finding container 92e1d5c2c3efdc155ba34a14af8bd8b51e79120da1e0c57b8d4ae96ce37a206d: Status 404 returned error can't find the container with id 92e1d5c2c3efdc155ba34a14af8bd8b51e79120da1e0c57b8d4ae96ce37a206d
Sep 07 09:33:22.912603 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.912499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt" event={"ID":"59841c33-ac6e-47df-b17c-ef86c59c08ad","Type":"ContainerStarted","Data":"fc5abcfeb046b39d80aadb7673be808e47917179bd708e7ab37e9422013f6bfd"}
Sep 07 09:33:22.912767 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.912605    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:33:22.912767 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.912629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt" event={"ID":"59841c33-ac6e-47df-b17c-ef86c59c08ad","Type":"ContainerStarted","Data":"92e1d5c2c3efdc155ba34a14af8bd8b51e79120da1e0c57b8d4ae96ce37a206d"}
Sep 07 09:33:22.935774 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:22.935719    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt" podStartSLOduration=1.935705198 podStartE2EDuration="1.935705198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:33:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:33:22.935477806 +0000 UTC m=+2060.904648411" watchObservedRunningTime="2026-09-07 09:33:22.935705198 +0000 UTC m=+2060.904875802"
Sep 07 09:33:33.919544 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:33:33.919509    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-rmgzt"
Sep 07 09:35:23.630181 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.630104    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"]
Sep 07 09:35:23.633489 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.633470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.644351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.644322    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"]
Sep 07 09:35:23.782172 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.782133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/072adb84-012c-4c2c-ae03-1fd22432c83d-maas-api-tls\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.782172 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.782169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/072adb84-012c-4c2c-ae03-1fd22432c83d-metrics-certs\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.782417 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.782193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/072adb84-012c-4c2c-ae03-1fd22432c83d-openshift-service-ca\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.782417 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.782270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n2g5\" (UniqueName: \"kubernetes.io/projected/072adb84-012c-4c2c-ae03-1fd22432c83d-kube-api-access-8n2g5\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.882973 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.882877    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/072adb84-012c-4c2c-ae03-1fd22432c83d-maas-api-tls\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.882973 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.882916    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/072adb84-012c-4c2c-ae03-1fd22432c83d-metrics-certs\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.882973 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.882938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/072adb84-012c-4c2c-ae03-1fd22432c83d-openshift-service-ca\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.883243 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.882989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n2g5\" (UniqueName: \"kubernetes.io/projected/072adb84-012c-4c2c-ae03-1fd22432c83d-kube-api-access-8n2g5\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.883737 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.883712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/072adb84-012c-4c2c-ae03-1fd22432c83d-openshift-service-ca\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.885611 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.885587    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/072adb84-012c-4c2c-ae03-1fd22432c83d-maas-api-tls\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.885725 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.885651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/072adb84-012c-4c2c-ae03-1fd22432c83d-metrics-certs\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.890895 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.890874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n2g5\" (UniqueName: \"kubernetes.io/projected/072adb84-012c-4c2c-ae03-1fd22432c83d-kube-api-access-8n2g5\") pod \"maas-api-7f77f9455f-8thnh\" (UID: \"072adb84-012c-4c2c-ae03-1fd22432c83d\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:23.945091 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:23.945059    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:24.214798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:24.214769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"]
Sep 07 09:35:24.216931 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:35:24.216906    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072adb84_012c_4c2c_ae03_1fd22432c83d.slice/crio-4c38eb4d86bc30e30775fb411951ef8d7958d9736afa9a0dc9a3a4057dafd955 WatchSource:0}: Error finding container 4c38eb4d86bc30e30775fb411951ef8d7958d9736afa9a0dc9a3a4057dafd955: Status 404 returned error can't find the container with id 4c38eb4d86bc30e30775fb411951ef8d7958d9736afa9a0dc9a3a4057dafd955
Sep 07 09:35:24.481241 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:24.481155    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh" event={"ID":"072adb84-012c-4c2c-ae03-1fd22432c83d","Type":"ContainerStarted","Data":"4c38eb4d86bc30e30775fb411951ef8d7958d9736afa9a0dc9a3a4057dafd955"}
Sep 07 09:35:25.488400 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:25.488358    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh" event={"ID":"072adb84-012c-4c2c-ae03-1fd22432c83d","Type":"ContainerStarted","Data":"878bce40d697db46cb718568f89bc602ee01e5e655e72763ac08df322ea80842"}
Sep 07 09:35:25.488800 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:25.488472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:25.507984 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:25.507927    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh" podStartSLOduration=2.067077852 podStartE2EDuration="2.50790696s" totalImagesPullingTime="440.829108ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:35:25.506623987 +0000 UTC m=+2183.475794592" watchObservedRunningTime="2026-09-07 09:35:25.50790696 +0000 UTC m=+2183.477077542"
Sep 07 09:35:31.498798 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:31.498764    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-8thnh"
Sep 07 09:35:31.545922 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:31.545882    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"]
Sep 07 09:35:31.546227 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:31.546199    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb" podUID="10ea0976-0725-4357-94ad-17791346359e" containerName="maas-api" containerID="cri-o://a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033" gracePeriod=30
Sep 07 09:35:32.799808 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.799782    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:35:32.873826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.873733    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zttkw\" (UniqueName: \"kubernetes.io/projected/10ea0976-0725-4357-94ad-17791346359e-kube-api-access-zttkw\") pod \"10ea0976-0725-4357-94ad-17791346359e\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") "
Sep 07 09:35:32.873826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.873783    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-metrics-certs\") pod \"10ea0976-0725-4357-94ad-17791346359e\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") "
Sep 07 09:35:32.873826 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.873805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-maas-api-tls\") pod \"10ea0976-0725-4357-94ad-17791346359e\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") "
Sep 07 09:35:32.874109 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.873849    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea0976-0725-4357-94ad-17791346359e-openshift-service-ca\") pod \"10ea0976-0725-4357-94ad-17791346359e\" (UID: \"10ea0976-0725-4357-94ad-17791346359e\") "
Sep 07 09:35:32.874352 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.874316    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10ea0976-0725-4357-94ad-17791346359e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10ea0976-0725-4357-94ad-17791346359e" (UID: "10ea0976-0725-4357-94ad-17791346359e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:35:32.875962 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.875926    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ea0976-0725-4357-94ad-17791346359e-kube-api-access-zttkw" (OuterVolumeSpecName: "kube-api-access-zttkw") pod "10ea0976-0725-4357-94ad-17791346359e" (UID: "10ea0976-0725-4357-94ad-17791346359e"). InnerVolumeSpecName "kube-api-access-zttkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:35:32.875962 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.875939    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10ea0976-0725-4357-94ad-17791346359e" (UID: "10ea0976-0725-4357-94ad-17791346359e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:35:32.876093 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.875956    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "10ea0976-0725-4357-94ad-17791346359e" (UID: "10ea0976-0725-4357-94ad-17791346359e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:35:32.974484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.974447    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-maas-api-tls\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:35:32.974484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.974476    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea0976-0725-4357-94ad-17791346359e-openshift-service-ca\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:35:32.974484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.974486    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zttkw\" (UniqueName: \"kubernetes.io/projected/10ea0976-0725-4357-94ad-17791346359e-kube-api-access-zttkw\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:35:32.974484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:32.974495    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ea0976-0725-4357-94ad-17791346359e-metrics-certs\") on node \"ip-10-0-138-157.ec2.internal\" DevicePath \"\""
Sep 07 09:35:33.528388 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:33.528352    2563 generic.go:356] "Generic (PLEG): container finished" podID="10ea0976-0725-4357-94ad-17791346359e" containerID="a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033" exitCode=0
Sep 07 09:35:33.528542 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:33.528413    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"
Sep 07 09:35:33.528542 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:33.528436    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb" event={"ID":"10ea0976-0725-4357-94ad-17791346359e","Type":"ContainerDied","Data":"a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033"}
Sep 07 09:35:33.528542 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:33.528495    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb" event={"ID":"10ea0976-0725-4357-94ad-17791346359e","Type":"ContainerDied","Data":"f8c17875af342817e5500664cbb2fc4e1a8313898da5db3ff3466e60e18a1dfb"}
Sep 07 09:35:33.528542 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:33.528518    2563 scope.go:117] "RemoveContainer" containerID="a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033"
Sep 07 09:35:33.538836 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:33.538751    2563 scope.go:117] "RemoveContainer" containerID="a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033"
Sep 07 09:35:33.540326 ip-10-0-138-157 kubenswrapper[2563]: E0907 09:35:33.540297    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033\": container with ID starting with a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033 not found: ID does not exist" containerID="a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033"
Sep 07 09:35:33.540427 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:33.540339    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033"} err="failed to get container status \"a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033\": rpc error: code = NotFound desc = could not find container \"a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033\": container with ID starting with a0d79cf2c152298884566aa319de54333f2b319626b0cd99c757f3c1eebdf033 not found: ID does not exist"
Sep 07 09:35:33.550003 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:33.549976    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"]
Sep 07 09:35:33.553672 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:33.553651    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-rvlhb"]
Sep 07 09:35:34.464538 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:34.464397    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10ea0976-0725-4357-94ad-17791346359e" path="/var/lib/kubelet/pods/10ea0976-0725-4357-94ad-17791346359e/volumes"
Sep 07 09:35:50.140721 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:35:50.140688    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5fcc76f457-qkpsr_b66812ef-287c-4024-b311-e00e51e7f23c/authorino/0.log"
Sep 07 09:36:00.397399 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:00.397365    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7xhnk_f7b195d2-b4fe-42a2-a39c-ac449c8c9af6/manager/0.log"
Sep 07 09:36:00.508953 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:00.508922    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6957b5f6b-kv72g_550e03a0-fe1a-4a1e-9dcc-93a526627f45/manager/0.log"
Sep 07 09:36:00.860699 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:00.860668    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-2bllj_4f294bc4-cd28-44e8-be90-1713cafe27cd/copy-manifests/0.log"
Sep 07 09:36:00.866681 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:00.866657    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-2bllj_4f294bc4-cd28-44e8-be90-1713cafe27cd/manager/0.log"
Sep 07 09:36:01.580946 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:01.580898    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7f77f9455f-8thnh_072adb84-012c-4c2c-ae03-1fd22432c83d/maas-api/0.log"
Sep 07 09:36:01.694247 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:01.694213    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-rgtqx_112badc2-95a3-448b-b7bd-1897bf55dbd2/postgres/0.log"
Sep 07 09:36:02.436331 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:02.436273    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5fcc76f457-qkpsr_b66812ef-287c-4024-b311-e00e51e7f23c/authorino/0.log"
Sep 07 09:36:02.881553 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:02.881524    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-rmgzt_59841c33-ac6e-47df-b17c-ef86c59c08ad/manager/0.log"
Sep 07 09:36:03.661484 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:03.661453    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b86566b9f-78vr6_52e01574-10fe-4d0f-b511-1b3ba9cb7e8b/kube-auth-proxy/0.log"
Sep 07 09:36:04.106801 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:04.106772    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-s55z8_6e49a562-3d44-40a0-a184-b90e5cd812dc/payload-processing/0.log"
Sep 07 09:36:10.164403 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.164363    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b5sms/must-gather-289xv"]
Sep 07 09:36:10.164857 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.164710    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ea0976-0725-4357-94ad-17791346359e" containerName="maas-api"
Sep 07 09:36:10.164857 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.164721    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ea0976-0725-4357-94ad-17791346359e" containerName="maas-api"
Sep 07 09:36:10.164857 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.164786    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="10ea0976-0725-4357-94ad-17791346359e" containerName="maas-api"
Sep 07 09:36:10.167844 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.167823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5sms/must-gather-289xv"
Sep 07 09:36:10.170542 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.170524    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b5sms\"/\"default-dockercfg-xmpz2\""
Sep 07 09:36:10.171105 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.171082    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b5sms\"/\"kube-root-ca.crt\""
Sep 07 09:36:10.171219 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.171135    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b5sms\"/\"openshift-service-ca.crt\""
Sep 07 09:36:10.176188 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.176166    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5sms/must-gather-289xv"]
Sep 07 09:36:10.312981 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.312940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9413e209-f414-45fd-b145-0631a48631a5-must-gather-output\") pod \"must-gather-289xv\" (UID: \"9413e209-f414-45fd-b145-0631a48631a5\") " pod="openshift-must-gather-b5sms/must-gather-289xv"
Sep 07 09:36:10.313168 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.312988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7mh7\" (UniqueName: \"kubernetes.io/projected/9413e209-f414-45fd-b145-0631a48631a5-kube-api-access-m7mh7\") pod \"must-gather-289xv\" (UID: \"9413e209-f414-45fd-b145-0631a48631a5\") " pod="openshift-must-gather-b5sms/must-gather-289xv"
Sep 07 09:36:10.413974 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.413917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9413e209-f414-45fd-b145-0631a48631a5-must-gather-output\") pod \"must-gather-289xv\" (UID: \"9413e209-f414-45fd-b145-0631a48631a5\") " pod="openshift-must-gather-b5sms/must-gather-289xv"
Sep 07 09:36:10.414170 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.413984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7mh7\" (UniqueName: \"kubernetes.io/projected/9413e209-f414-45fd-b145-0631a48631a5-kube-api-access-m7mh7\") pod \"must-gather-289xv\" (UID: \"9413e209-f414-45fd-b145-0631a48631a5\") " pod="openshift-must-gather-b5sms/must-gather-289xv"
Sep 07 09:36:10.414312 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.414262    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9413e209-f414-45fd-b145-0631a48631a5-must-gather-output\") pod \"must-gather-289xv\" (UID: \"9413e209-f414-45fd-b145-0631a48631a5\") " pod="openshift-must-gather-b5sms/must-gather-289xv"
Sep 07 09:36:10.422067 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.422013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7mh7\" (UniqueName: \"kubernetes.io/projected/9413e209-f414-45fd-b145-0631a48631a5-kube-api-access-m7mh7\") pod \"must-gather-289xv\" (UID: \"9413e209-f414-45fd-b145-0631a48631a5\") " pod="openshift-must-gather-b5sms/must-gather-289xv"
Sep 07 09:36:10.477182 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.477149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5sms/must-gather-289xv"
Sep 07 09:36:10.751220 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.751185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5sms/must-gather-289xv"]
Sep 07 09:36:10.754751 ip-10-0-138-157 kubenswrapper[2563]: W0907 09:36:10.754727    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9413e209_f414_45fd_b145_0631a48631a5.slice/crio-2058445f529f15e1b45f80513dfdfe7e0efb85472ecc1055155486057f86cbcf WatchSource:0}: Error finding container 2058445f529f15e1b45f80513dfdfe7e0efb85472ecc1055155486057f86cbcf: Status 404 returned error can't find the container with id 2058445f529f15e1b45f80513dfdfe7e0efb85472ecc1055155486057f86cbcf
Sep 07 09:36:10.756746 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:10.756728    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:36:11.704157 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:11.704117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5sms/must-gather-289xv" event={"ID":"9413e209-f414-45fd-b145-0631a48631a5","Type":"ContainerStarted","Data":"8ee12c1a7e8bcf50ba5f330e6f6a45b20b60610ec4a57d459d72628df5a97a21"}
Sep 07 09:36:11.704668 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:11.704174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5sms/must-gather-289xv" event={"ID":"9413e209-f414-45fd-b145-0631a48631a5","Type":"ContainerStarted","Data":"2058445f529f15e1b45f80513dfdfe7e0efb85472ecc1055155486057f86cbcf"}
Sep 07 09:36:12.712085 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:12.712030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5sms/must-gather-289xv" event={"ID":"9413e209-f414-45fd-b145-0631a48631a5","Type":"ContainerStarted","Data":"fee2e0357987f8e7129650c16ef160c7fab3792e17908714b963619e0e84a7db"}
Sep 07 09:36:12.728873 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:12.728821    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b5sms/must-gather-289xv" podStartSLOduration=1.929410571 podStartE2EDuration="2.728805506s" totalImagesPullingTime="799.394935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:36:12.725643352 +0000 UTC m=+2230.694813957" watchObservedRunningTime="2026-09-07 09:36:12.728805506 +0000 UTC m=+2230.697976145"
Sep 07 09:36:13.037586 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:13.037550    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xpfh7_da4c2b44-97ef-47ec-812b-6f3bb1b5c536/global-pull-secret-syncer/0.log"
Sep 07 09:36:13.136123 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:13.136092    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m6gv9_2767bb9a-644e-4a87-999c-a90f8916437b/konnectivity-agent/0.log"
Sep 07 09:36:13.199867 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:13.199837    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-157.ec2.internal_34b1847e43a3aeca49fcecd34c67497c/haproxy/0.log"
Sep 07 09:36:17.593307 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:17.591799    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5fcc76f457-qkpsr_b66812ef-287c-4024-b311-e00e51e7f23c/authorino/0.log"
Sep 07 09:36:17.733260 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:17.733224    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-rmgzt_59841c33-ac6e-47df-b17c-ef86c59c08ad/manager/0.log"
Sep 07 09:36:19.312518 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.312488    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfd83436-230a-4e51-b199-ca5640e31222/alertmanager/0.log"
Sep 07 09:36:19.334533 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.334452    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfd83436-230a-4e51-b199-ca5640e31222/config-reloader/0.log"
Sep 07 09:36:19.360177 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.360151    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfd83436-230a-4e51-b199-ca5640e31222/kube-rbac-proxy-web/0.log"
Sep 07 09:36:19.382557 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.382528    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfd83436-230a-4e51-b199-ca5640e31222/kube-rbac-proxy/0.log"
Sep 07 09:36:19.405091 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.405064    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfd83436-230a-4e51-b199-ca5640e31222/kube-rbac-proxy-metric/0.log"
Sep 07 09:36:19.426048 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.426012    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfd83436-230a-4e51-b199-ca5640e31222/prom-label-proxy/0.log"
Sep 07 09:36:19.446624 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.446592    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bfd83436-230a-4e51-b199-ca5640e31222/init-config-reloader/0.log"
Sep 07 09:36:19.590407 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.590328    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79b8f874c5-5dq4f_73de73c6-b550-41e0-a5d4-8e921102b155/metrics-server/0.log"
Sep 07 09:36:19.625342 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.625313    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-rq2rh_c3b71592-f725-4636-8ada-7e6a51564fcc/monitoring-plugin/0.log"
Sep 07 09:36:19.655582 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.655553    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9xjcg_452b70d9-3fa4-49a4-a65b-5688bdef59c0/node-exporter/0.log"
Sep 07 09:36:19.686419 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.686392    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9xjcg_452b70d9-3fa4-49a4-a65b-5688bdef59c0/kube-rbac-proxy/0.log"
Sep 07 09:36:19.706704 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.706677    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9xjcg_452b70d9-3fa4-49a4-a65b-5688bdef59c0/init-textfile/0.log"
Sep 07 09:36:19.960581 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.960554    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5fde8ee7-0e57-4f22-858f-f5889b135e20/prometheus/0.log"
Sep 07 09:36:19.986105 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:19.985937    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5fde8ee7-0e57-4f22-858f-f5889b135e20/config-reloader/0.log"
Sep 07 09:36:20.002655 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.002624    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5fde8ee7-0e57-4f22-858f-f5889b135e20/thanos-sidecar/0.log"
Sep 07 09:36:20.021806 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.021751    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5fde8ee7-0e57-4f22-858f-f5889b135e20/kube-rbac-proxy-web/0.log"
Sep 07 09:36:20.041214 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.041184    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5fde8ee7-0e57-4f22-858f-f5889b135e20/kube-rbac-proxy/0.log"
Sep 07 09:36:20.060443 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.060361    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5fde8ee7-0e57-4f22-858f-f5889b135e20/kube-rbac-proxy-thanos/0.log"
Sep 07 09:36:20.079444 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.079418    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5fde8ee7-0e57-4f22-858f-f5889b135e20/init-config-reloader/0.log"
Sep 07 09:36:20.303646 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.303584    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-789cbcdfc6-drpk5_bc14e598-469a-4994-b597-583395e97815/thanos-query/0.log"
Sep 07 09:36:20.324880 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.324853    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-789cbcdfc6-drpk5_bc14e598-469a-4994-b597-583395e97815/kube-rbac-proxy-web/0.log"
Sep 07 09:36:20.347880 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.347831    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-789cbcdfc6-drpk5_bc14e598-469a-4994-b597-583395e97815/kube-rbac-proxy/0.log"
Sep 07 09:36:20.373612 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.373577    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-789cbcdfc6-drpk5_bc14e598-469a-4994-b597-583395e97815/prom-label-proxy/0.log"
Sep 07 09:36:20.417306 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.416963    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-789cbcdfc6-drpk5_bc14e598-469a-4994-b597-583395e97815/kube-rbac-proxy-rules/0.log"
Sep 07 09:36:20.450649 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:20.450608    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-789cbcdfc6-drpk5_bc14e598-469a-4994-b597-583395e97815/kube-rbac-proxy-metrics/0.log"
Sep 07 09:36:23.848016 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:23.847972    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"]
Sep 07 09:36:23.855637 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:23.855603    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:23.862947 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:23.862909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"]
Sep 07 09:36:23.956415 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:23.956379    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-sys\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:23.956616 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:23.956455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-podres\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:23.956616 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:23.956513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-lib-modules\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:23.956616 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:23.956573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pgmd\" (UniqueName: \"kubernetes.io/projected/d98f83f4-68ba-4008-bf5e-b7ae8147a777-kube-api-access-5pgmd\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:23.956808 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:23.956707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-proc\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.057683 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.057644    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-proc\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.057956 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.057939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-sys\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.058104 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.058090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-podres\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.058247 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.058233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-lib-modules\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.058399 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.058384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pgmd\" (UniqueName: \"kubernetes.io/projected/d98f83f4-68ba-4008-bf5e-b7ae8147a777-kube-api-access-5pgmd\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.058556 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.058441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-lib-modules\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.058720 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.058704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-sys\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.058818 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.057842    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-proc\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.058942 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.058921    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d98f83f4-68ba-4008-bf5e-b7ae8147a777-podres\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.067558 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.067536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pgmd\" (UniqueName: \"kubernetes.io/projected/d98f83f4-68ba-4008-bf5e-b7ae8147a777-kube-api-access-5pgmd\") pod \"perf-node-gather-daemonset-8d6lz\" (UID: \"d98f83f4-68ba-4008-bf5e-b7ae8147a777\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.177450 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.177370    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.472031 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.471955    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"]
Sep 07 09:36:24.800883 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.800835    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz" event={"ID":"d98f83f4-68ba-4008-bf5e-b7ae8147a777","Type":"ContainerStarted","Data":"ec566c5e20218e7fc79c0aec89983b9d62bad1395361408f2e192a2d631b33d3"}
Sep 07 09:36:24.801083 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.800944    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:24.801083 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.800974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz" event={"ID":"d98f83f4-68ba-4008-bf5e-b7ae8147a777","Type":"ContainerStarted","Data":"70eb8954a8a2d070a5493f6cdf121ee96a1c79461b3149c922247700a5212093"}
Sep 07 09:36:24.816838 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:24.816775    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz" podStartSLOduration=1.816753947 podStartE2EDuration="1.816753947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 09:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:36:24.814575893 +0000 UTC m=+2242.783746499" watchObservedRunningTime="2026-09-07 09:36:24.816753947 +0000 UTC m=+2242.785924553"
Sep 07 09:36:25.255235 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:25.255203    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z7tq5_2b4d281b-4835-48b0-b6ba-6f558f113bbc/dns/0.log"
Sep 07 09:36:25.272082 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:25.272051    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z7tq5_2b4d281b-4835-48b0-b6ba-6f558f113bbc/kube-rbac-proxy/0.log"
Sep 07 09:36:25.294715 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:25.294684    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fkt8f_83680078-ce21-45b5-b66c-c4211b91f06b/dns-node-resolver/0.log"
Sep 07 09:36:25.843217 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:25.843180    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d4c6c4978-zbzln_809d141e-6429-49f6-b9a7-f3bcf0179714/registry/0.log"
Sep 07 09:36:25.880761 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:25.880726    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bw5zk_2c8d36ac-5cab-446f-8e12-19f48927540c/node-ca/0.log"
Sep 07 09:36:27.107371 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:27.107338    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b86566b9f-78vr6_52e01574-10fe-4d0f-b511-1b3ba9cb7e8b/kube-auth-proxy/0.log"
Sep 07 09:36:27.236064 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:27.235982    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-s55z8_6e49a562-3d44-40a0-a184-b90e5cd812dc/payload-processing/0.log"
Sep 07 09:36:28.116701 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:28.116672    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jvwmj_bb78a497-2227-45ef-bd86-fc1cdece6bae/serve-healthcheck-canary/0.log"
Sep 07 09:36:28.637648 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:28.637620    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n2qsg_f50474e9-671c-414f-987d-50491a4978eb/kube-rbac-proxy/0.log"
Sep 07 09:36:28.656042 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:28.656013    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n2qsg_f50474e9-671c-414f-987d-50491a4978eb/exporter/0.log"
Sep 07 09:36:28.673604 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:28.673568    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n2qsg_f50474e9-671c-414f-987d-50491a4978eb/extractor/0.log"
Sep 07 09:36:30.820903 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:30.820854    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-8d6lz"
Sep 07 09:36:31.017696 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:31.017662    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7xhnk_f7b195d2-b4fe-42a2-a39c-ac449c8c9af6/manager/0.log"
Sep 07 09:36:31.076351 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:31.073264    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6957b5f6b-kv72g_550e03a0-fe1a-4a1e-9dcc-93a526627f45/manager/0.log"
Sep 07 09:36:31.189920 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:31.189892    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-2bllj_4f294bc4-cd28-44e8-be90-1713cafe27cd/manager/0.log"
Sep 07 09:36:31.207067 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:31.207043    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-2bllj_4f294bc4-cd28-44e8-be90-1713cafe27cd/copy-manifests/0.log"
Sep 07 09:36:32.583141 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:32.583108    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-94f7b5c54-m9zcl_fef8b845-166b-4e83-aa00-fbdbc49cdea6/manager/0.log"
Sep 07 09:36:34.187071 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:34.187043    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b86566b9f-78vr6_52e01574-10fe-4d0f-b511-1b3ba9cb7e8b/kube-auth-proxy/0.log"
Sep 07 09:36:34.283155 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:34.283118    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-s55z8_6e49a562-3d44-40a0-a184-b90e5cd812dc/payload-processing/0.log"
Sep 07 09:36:36.148888 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:36.148846    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b86566b9f-78vr6_52e01574-10fe-4d0f-b511-1b3ba9cb7e8b/kube-auth-proxy/0.log"
Sep 07 09:36:36.249379 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:36.249266    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-s55z8_6e49a562-3d44-40a0-a184-b90e5cd812dc/payload-processing/0.log"
Sep 07 09:36:38.912930 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:38.912906    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdspn_b06f9c26-d0eb-4e4d-8424-51087b91fa18/kube-multus-additional-cni-plugins/0.log"
Sep 07 09:36:38.930394 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:38.930369    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdspn_b06f9c26-d0eb-4e4d-8424-51087b91fa18/egress-router-binary-copy/0.log"
Sep 07 09:36:38.949482 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:38.949391    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdspn_b06f9c26-d0eb-4e4d-8424-51087b91fa18/cni-plugins/0.log"
Sep 07 09:36:38.967799 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:38.967768    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdspn_b06f9c26-d0eb-4e4d-8424-51087b91fa18/bond-cni-plugin/0.log"
Sep 07 09:36:38.986061 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:38.986036    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdspn_b06f9c26-d0eb-4e4d-8424-51087b91fa18/routeoverride-cni/0.log"
Sep 07 09:36:39.003219 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:39.003194    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdspn_b06f9c26-d0eb-4e4d-8424-51087b91fa18/whereabouts-cni-bincopy/0.log"
Sep 07 09:36:39.020547 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:39.020514    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fdspn_b06f9c26-d0eb-4e4d-8424-51087b91fa18/whereabouts-cni/0.log"
Sep 07 09:36:39.344182 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:39.344151    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tg9lx_53d15ca2-406b-450f-9fc2-196927235f32/kube-multus/0.log"
Sep 07 09:36:39.402022 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:39.401985    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bmllm_79f55277-100d-44e8-b8e4-a1ca8c9ed7e5/network-metrics-daemon/0.log"
Sep 07 09:36:39.416141 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:39.416108    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bmllm_79f55277-100d-44e8-b8e4-a1ca8c9ed7e5/kube-rbac-proxy/0.log"
Sep 07 09:36:40.413355 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:40.413321    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vvtcs_762acdac-34ca-469a-87e9-e945fb096c34/ovn-controller/0.log"
Sep 07 09:36:40.432052 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:40.432031    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vvtcs_762acdac-34ca-469a-87e9-e945fb096c34/ovn-acl-logging/0.log"
Sep 07 09:36:40.450201 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:40.450177    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vvtcs_762acdac-34ca-469a-87e9-e945fb096c34/kube-rbac-proxy-node/0.log"
Sep 07 09:36:40.479957 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:40.479933    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vvtcs_762acdac-34ca-469a-87e9-e945fb096c34/kube-rbac-proxy-ovn-metrics/0.log"
Sep 07 09:36:40.499380 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:40.499349    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vvtcs_762acdac-34ca-469a-87e9-e945fb096c34/northd/0.log"
Sep 07 09:36:40.516462 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:40.516436    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vvtcs_762acdac-34ca-469a-87e9-e945fb096c34/nbdb/0.log"
Sep 07 09:36:40.533938 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:40.533913    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vvtcs_762acdac-34ca-469a-87e9-e945fb096c34/sbdb/0.log"
Sep 07 09:36:40.591363 ip-10-0-138-157 kubenswrapper[2563]: I0907 09:36:40.591332    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vvtcs_762acdac-34ca-469a-87e9-e945fb096c34/ovnkube-controller/0.log"