Sep 11 18:15:36.854052 ip-10-0-141-82 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 18:15:37.248136 ip-10-0-141-82 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 18:15:37.248136 ip-10-0-141-82 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 18:15:37.248136 ip-10-0-141-82 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 18:15:37.248136 ip-10-0-141-82 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 18:15:37.248136 ip-10-0-141-82 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 18:15:37.249785 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.249694    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 18:15:37.252773 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252758    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:15:37.252773 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252771    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:15:37.252773 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252776    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252779    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252784    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252789    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252792    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252796    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252799    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252801    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252809    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252812    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252814    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252817    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252820    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252823    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252825    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252828    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252830    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252833    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252836    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:15:37.252864 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252838    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252841    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252843    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252847    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252851    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252854    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252857    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252859    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252862    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252865    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252868    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252872    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252874    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252877    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252880    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252884    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252887    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252890    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252893    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252896    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:15:37.253312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252898    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252901    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252903    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252906    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252908    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252911    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252913    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252916    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252918    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252921    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252923    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252926    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252928    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252931    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252934    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252937    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252940    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252943    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252945    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252948    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:15:37.253813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252950    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252954    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252958    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252961    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252964    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252966    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252969    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252971    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252974    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252976    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252979    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252982    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252984    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252987    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252989    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252993    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252996    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.252998    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253001    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253003    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:15:37.254307 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253006    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253008    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253011    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253013    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253016    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253018    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253022    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253024    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253027    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253030    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253033    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253036    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253038    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253041    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253044    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253046    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253049    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253051    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253726    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253733    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:15:37.254845 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253736    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253739    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253743    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253752    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253755    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253757    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253760    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253763    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253765    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253768    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253770    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253773    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253775    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253778    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253781    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253783    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253786    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253788    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253791    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253794    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:15:37.255333 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253797    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253800    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253803    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253807    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253811    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253814    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253818    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253821    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253825    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253827    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253830    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253833    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253836    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253838    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253841    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253845    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253854    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253857    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253859    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253862    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:15:37.255839 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253865    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253868    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253870    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253872    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253875    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253878    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253880    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253883    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253885    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253888    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253892    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253895    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253898    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253901    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253903    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253906    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253908    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253911    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253914    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:15:37.256328 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253917    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253920    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253922    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253925    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253927    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253930    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253933    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253935    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253938    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253940    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253948    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253952    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253956    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253959    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253961    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253964    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253968    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253971    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253974    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:15:37.256801 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253976    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253979    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253982    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253984    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253987    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253990    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253993    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253996    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.253998    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254001    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254003    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254006    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254008    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254011    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254013    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254016    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254018    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254021    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254023    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254385    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:15:37.257260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254391    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254395    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254398    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254401    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254404    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254407    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254410    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254413    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254415    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254419    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254421    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254424    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254427    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254429    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254432    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254434    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254437    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254440    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254442    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254445    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:15:37.257759 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254448    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254450    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254453    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254455    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254458    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254461    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254464    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254467    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254469    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254471    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254474    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254476    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254479    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254483    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254486    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254489    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254492    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254495    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254499    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:15:37.258256 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254502    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254505    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254508    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254511    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254514    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254516    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254519    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254521    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254524    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254526    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254529    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254531    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254535    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254539    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254542    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254545    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254547    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254550    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254553    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:15:37.258740 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254556    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254558    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254561    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254564    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254566    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254568    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254571    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254574    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254576    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254579    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254581    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254585    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254587    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254590    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254593    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254596    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254598    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254601    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254603    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254606    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254608    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:15:37.259204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254611    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254613    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254616    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254618    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254621    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254623    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254626    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254629    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254631    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254634    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254636    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254639    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254641    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254645    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254650    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254653    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254655    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254658    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.254661    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254756    2578 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254793    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 18:15:37.259771 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254800    2578 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254805    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254810    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254813    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254818    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254822    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254825    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254829    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254832    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254835    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254838    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254841    2578 flags.go:64] FLAG: --cgroup-root=""
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254844    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254847    2578 flags.go:64] FLAG: --client-ca-file=""
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254850    2578 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254853    2578 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254858    2578 flags.go:64] FLAG: --cluster-domain=""
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254862    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254865    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254868    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254871    2578 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254880    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254883    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 18:15:37.260291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254886    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254889    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254894    2578 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254897    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254900    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254903    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254905    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254910    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254912    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254915    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254918    2578 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254921    2578 flags.go:64] FLAG: --enable-server="true"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254925    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254929    2578 flags.go:64] FLAG: --event-burst="100"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254932    2578 flags.go:64] FLAG: --event-qps="50"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254935    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254938    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254941    2578 flags.go:64] FLAG: --eviction-hard=""
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254944    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254947    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254950    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254953    2578 flags.go:64] FLAG: --eviction-soft=""
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254956    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254958    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254961    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 18:15:37.260871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254964    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254967    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254970    2578 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254973    2578 flags.go:64] FLAG: --feature-gates=""
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254977    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254981    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254984    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254987    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254991    2578 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254995    2578 flags.go:64] FLAG: --help="false"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.254998    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255001    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255004    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255007    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255010    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255014    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255016    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255019    2578 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255022    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255025    2578 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255028    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255031    2578 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255034    2578 flags.go:64] FLAG: --kube-reserved=""
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255037    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 18:15:37.261488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255040    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255043    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255045    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255048    2578 flags.go:64] FLAG: --lock-file=""
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255051    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255054    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255057    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255062    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255065    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255068    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255070    2578 flags.go:64] FLAG: --logging-format="text"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255073    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255076    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255080    2578 flags.go:64] FLAG: --manifest-url=""
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255083    2578 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255087    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255090    2578 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255096    2578 flags.go:64] FLAG: --max-pods="110"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255099    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255102    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255105    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255108    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255111    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255114    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255116    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 18:15:37.262114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255123    2578 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255126    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255129    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255132    2578 flags.go:64] FLAG: --pod-cidr=""
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255135    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255140    2578 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255143    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255146    2578 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255149    2578 flags.go:64] FLAG: --port="10250"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255152    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255155    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-012588d176d395246"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255158    2578 flags.go:64] FLAG: --qos-reserved=""
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255161    2578 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255164    2578 flags.go:64] FLAG: --register-node="true"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255167    2578 flags.go:64] FLAG: --register-with-taints=""
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255171    2578 flags.go:64] FLAG: --registry-burst="10"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255173    2578 flags.go:64] FLAG: --registry-qps="5"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255176    2578 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255179    2578 flags.go:64] FLAG: --reserved-memory=""
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255185    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255190    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255193    2578 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255196    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255199    2578 flags.go:64] FLAG: --runonce="false"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255202    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 18:15:37.262786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255206    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255209    2578 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255212    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255215    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255218    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255221    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255224    2578 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255227    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255230    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255233    2578 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255236    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255239    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255242    2578 flags.go:64] FLAG: --system-cgroups=""
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255245    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255250    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255253    2578 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255256    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255260    2578 flags.go:64] FLAG: --tls-min-version=""
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255263    2578 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255266    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255269    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255271    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255274    2578 flags.go:64] FLAG: --v="2"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255278    2578 flags.go:64] FLAG: --version="false"
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255282    2578 flags.go:64] FLAG: --vmodule=""
Sep 11 18:15:37.263493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255285    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255289    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255408    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255411    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255415    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255418    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255421    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255426    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255430    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255433    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255436    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255439    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255442    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255444    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255447    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255450    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255453    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255455    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255458    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255461    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:15:37.264145 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255464    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255467    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255469    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255472    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255475    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255478    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255480    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255483    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255486    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255488    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255491    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255493    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255496    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255498    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255502    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255505    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255508    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255510    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255513    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255517    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:15:37.264641 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255519    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255522    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255525    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255527    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255530    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255533    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255535    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255539    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255543    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255546    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255548    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255551    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255554    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255557    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255559    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255562    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255565    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255567    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255570    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:15:37.265157 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255572    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255575    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255579    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255582    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255585    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255588    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255591    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255594    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255597    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255599    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255602    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255604    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255609    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255612    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255614    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255617    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255620    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255622    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255625    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:15:37.265619 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255628    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255630    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255633    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255635    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255638    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255641    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255643    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255646    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255649    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255651    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255654    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255657    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255659    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255662    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255664    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255667    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255669    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255672    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255689    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255693    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:15:37.266103 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255697    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255700    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.255703    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.255709    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.264044    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.264059    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.264937    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.264945    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265002    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265007    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265010    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265014    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265017    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265019    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265022    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:15:37.266643 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265025    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265028    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265031    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265035    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265038    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265042    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265044    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265047    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265050    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265052    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265055    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265058    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265060    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265063    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265066    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265069    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265071    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265074    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265076    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:15:37.267038 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265079    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265081    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265085    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265088    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265091    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265093    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265096    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265099    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265102    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265104    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265107    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265110    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265113    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265115    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265118    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265121    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265123    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265126    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265129    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265132    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:15:37.267494 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265135    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265137    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265140    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265142    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265145    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265148    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265151    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265153    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265156    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265159    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265161    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265164    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265167    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265170    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265173    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265178    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265182    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265185    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265188    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265191    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:15:37.268009 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265194    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265196    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265199    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265202    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265205    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265207    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265210    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265214    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265217    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265220    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265222    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265225    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265229    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265231    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265234    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265237    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265239    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265243    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265246    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265248    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:15:37.268515 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265251    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265253    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265256    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265258    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265261    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265263    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265266    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265268    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265271    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265274    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265276    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265279    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265282    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.265287    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265398    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:15:37.269019 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265403    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265406    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265409    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265412    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265415    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265418    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265420    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265423    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265426    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265428    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265431    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265434    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265436    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265439    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265442    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265445    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265447    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265450    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265452    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:15:37.269391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265455    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265457    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265460    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265462    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265465    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265468    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265470    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265473    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265476    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265478    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265481    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265484    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265486    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265489    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265493    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265496    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265499    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265502    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265504    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:15:37.269872 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265507    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265510    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265512    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265515    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265517    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265520    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265523    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265525    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265528    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265532    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265536    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265539    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265542    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265545    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265547    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265550    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265553    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265556    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265559    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:15:37.270349 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265562    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265564    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265567    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265570    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265572    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265575    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265577    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265580    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265582    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265585    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265587    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265590    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265592    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265595    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265597    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265600    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265602    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265605    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265607    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265610    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:15:37.270824 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265612    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265616    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265618    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265621    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265624    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265626    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265629    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265632    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265634    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265637    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265639    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265642    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265645    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265647    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265651    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265654    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265657    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265660    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265663    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265666    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:15:37.271300 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:37.265668    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:15:37.271825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.265673    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 18:15:37.271825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.265782    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 18:15:37.271825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.267901    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 18:15:37.271825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.268832    2578 server.go:1019] "Starting client certificate rotation"
Sep 11 18:15:37.271825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.268933    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 18:15:37.271825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.268962    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 18:15:37.271825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.271717    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 18:15:37.279707 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.279668    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 18:15:37.291407 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.291389    2578 log.go:25] "Validated CRI v1 runtime API"
Sep 11 18:15:37.294547 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.294530    2578 log.go:25] "Validated CRI v1 image API"
Sep 11 18:15:37.296236 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.296220    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 18:15:37.299435 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.299417    2578 fs.go:135] Filesystem UUIDs: map[4189a608-6636-409c-bc31-9175bfd0e179:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 90be677f-1f72-419f-97ec-e888229ec1d2:/dev/nvme0n1p3]
Sep 11 18:15:37.299495 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.299435    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 18:15:37.304306 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.304287    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 18:15:37.304558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.304452    2578 manager.go:216] Machine: {Timestamp:2026-09-11 18:15:37.303006578 +0000 UTC m=+0.338223239 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3116829 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2913ade41414c6459cd6572b9f6854 SystemUUID:ec2913ad-e414-14c6-459c-d6572b9f6854 BootID:c74155dc-c071-4286-9539-aa5c02f94503 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:04:56:02:ec:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:04:56:02:ec:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:a2:86:7d:0b:14 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 18:15:37.304558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.304553    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 18:15:37.304652 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.304631    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 18:15:37.307770 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.307739    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 18:15:37.307936 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.307777    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-82.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 18:15:37.308417 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.308406    2578 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 18:15:37.308477 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.308424    2578 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 18:15:37.308477 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.308434    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 18:15:37.309086 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.309072    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 18:15:37.310233 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.310221    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 18:15:37.313822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.313808    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 18:15:37.313979 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.313968    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 18:15:37.314050 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.314005    2578 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 18:15:37.314050 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.314020    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 18:15:37.314050 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.314039    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 18:15:37.314173 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.314052    2578 kubelet.go:398] "Adding apiserver pod source"
Sep 11 18:15:37.314173 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.314065    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 18:15:37.314441 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.314427    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 18:15:37.314501 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.314449    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 18:15:37.316386 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.316365    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 18:15:37.320200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.320184    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 18:15:37.320313 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.320215    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 18:15:37.322076 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322063    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 18:15:37.322109 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322081    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 18:15:37.322109 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322087    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 18:15:37.322109 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322092    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 18:15:37.322109 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322100    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 18:15:37.322109 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322105    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 18:15:37.322251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322118    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 18:15:37.322251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322123    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 18:15:37.322251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322129    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 18:15:37.322251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322135    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 18:15:37.322251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322143    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 18:15:37.322251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322149    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 18:15:37.322931 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322918    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 18:15:37.322961 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.322936    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 18:15:37.324951 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.324933    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-82.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 18:15:37.326749 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.326736    2578 server.go:1268] "Started kubelet"
Sep 11 18:15:37.326841 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.326810    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 18:15:37.326933 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.326843    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 18:15:37.326933 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.326910    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 18:15:37.327536 ip-10-0-141-82 systemd[1]: Started Kubernetes Kubelet.
Sep 11 18:15:37.330102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.329097    2578 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 18:15:37.330102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.329804    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 18:15:37.332459 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.332434    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 18:15:37.333061 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.333037    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 18:15:37.333061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.333049    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-82.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 18:15:37.333335 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.333319    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wdjx5"
Sep 11 18:15:37.333998 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.333100    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-82.ec2.internal.18d456ef330ec51f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-82.ec2.internal,UID:ip-10-0-141-82.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-82.ec2.internal,},FirstTimestamp:2026-09-11 18:15:37.326707999 +0000 UTC m=+0.361924659,LastTimestamp:2026-09-11 18:15:37.326707999 +0000 UTC m=+0.361924659,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-82.ec2.internal,}"
Sep 11 18:15:37.335080 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.335060    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 18:15:37.335595 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.335580    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 18:15:37.335724 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.335709    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:37.336545 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.336527    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 18:15:37.336692 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.336660    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 18:15:37.336692 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.336692    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 18:15:37.336860 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.336847    2578 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 18:15:37.336860 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.336858    2578 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 18:15:37.337057 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.337040    2578 factory.go:153] Registering CRI-O factory
Sep 11 18:15:37.337057 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.337056    2578 factory.go:223] Registration of the crio container factory successfully
Sep 11 18:15:37.337161 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.337107    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 18:15:37.337161 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.337117    2578 factory.go:55] Registering systemd factory
Sep 11 18:15:37.337161 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.337124    2578 factory.go:223] Registration of the systemd container factory successfully
Sep 11 18:15:37.337161 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.337139    2578 factory.go:103] Registering Raw factory
Sep 11 18:15:37.337161 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.337152    2578 manager.go:1196] Started watching for new ooms in manager
Sep 11 18:15:37.338080 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.338064    2578 manager.go:349] Starting recovery of all containers
Sep 11 18:15:37.338199 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.338178    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-82.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 18:15:37.338376 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.338353    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 18:15:37.339590 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.339563    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wdjx5"
Sep 11 18:15:37.349184 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.348857    2578 manager.go:354] Recovery completed
Sep 11 18:15:37.354366 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.354353    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:15:37.356492 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.356476    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:15:37.356569 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.356509    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:15:37.356569 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.356526    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:15:37.357140 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.357127    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 18:15:37.357140 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.357138    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 18:15:37.357234 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.357155    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 18:15:37.357234 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.357137    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:15:37.357951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.357936    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:15:37.358005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.357962    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:15:37.358005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.357976    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:15:37.358393 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.358380    2578 policy_none.go:49] "None policy: Start"
Sep 11 18:15:37.358449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.358399    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 18:15:37.358449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.358413    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 18:15:37.359553 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.359540    2578 policy_none.go:47] "Start"
Sep 11 18:15:37.383667 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.383649    2578 manager.go:337] "Starting Device Plugin manager"
Sep 11 18:15:37.383801 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.383705    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 18:15:37.383801 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.383721    2578 server.go:87] "Starting device plugin registration server"
Sep 11 18:15:37.383999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.383984    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 18:15:37.384099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.383999    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 18:15:37.384188 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.384175    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 18:15:37.384267 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.384248    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 18:15:37.384310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.384269    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 18:15:37.384816 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.384794    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 18:15:37.384874 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.384852    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:37.402036 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.402004    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 18:15:37.407619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.403211    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 18:15:37.407619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.403262    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 18:15:37.407619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.403286    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 18:15:37.407619 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.403328    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 18:15:37.407619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.405590    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:15:37.484970 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.484946    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:15:37.486917 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.486895    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:15:37.487009 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.486935    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:15:37.487009 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.486951    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:15:37.487009 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.486980    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.502267 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.502218    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.502267 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.502242    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-82.ec2.internal\": node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:37.503443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.503429    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal"]
Sep 11 18:15:37.503497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.503488    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:15:37.504880 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.504844    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:15:37.504880 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.504871    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:15:37.504880 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.504883    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:15:37.506081 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.506070    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:15:37.506209 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.506195    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.506256 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.506229    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:15:37.506944 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.506910    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:15:37.506944 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.506937    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:15:37.507073 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.506951    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:15:37.507141 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.507122    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:15:37.507188 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.507151    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:15:37.507188 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.507161    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:15:37.508425 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.508406    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.508517 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.508433    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:15:37.510137 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.510119    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:15:37.510232 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.510149    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:15:37.510232 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.510171    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:15:37.520974 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.520954    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:37.538719 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.538689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b28de82e9dd393a989620953488acea7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal\" (UID: \"b28de82e9dd393a989620953488acea7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.538799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.538727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e205fce0f56018a4d3daccefe7a004e7-config\") pod \"kube-apiserver-proxy-ip-10-0-141-82.ec2.internal\" (UID: \"e205fce0f56018a4d3daccefe7a004e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.538799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.538745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b28de82e9dd393a989620953488acea7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal\" (UID: \"b28de82e9dd393a989620953488acea7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.543255 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.543237    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-82.ec2.internal\" not found" node="ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.547496 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.547480    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-82.ec2.internal\" not found" node="ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.621782 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.621597    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:37.639580 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.639557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e205fce0f56018a4d3daccefe7a004e7-config\") pod \"kube-apiserver-proxy-ip-10-0-141-82.ec2.internal\" (UID: \"e205fce0f56018a4d3daccefe7a004e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.639580 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.639581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b28de82e9dd393a989620953488acea7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal\" (UID: \"b28de82e9dd393a989620953488acea7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.639721 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.639597    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b28de82e9dd393a989620953488acea7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal\" (UID: \"b28de82e9dd393a989620953488acea7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.639721 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.639629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b28de82e9dd393a989620953488acea7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal\" (UID: \"b28de82e9dd393a989620953488acea7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.639721 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.639646    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e205fce0f56018a4d3daccefe7a004e7-config\") pod \"kube-apiserver-proxy-ip-10-0-141-82.ec2.internal\" (UID: \"e205fce0f56018a4d3daccefe7a004e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.639721 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.639670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b28de82e9dd393a989620953488acea7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal\" (UID: \"b28de82e9dd393a989620953488acea7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.722897 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.722868    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:37.824010 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.823937    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:37.845453 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.845431    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.851335 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:37.851312    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:37.924444 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:37.924411    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:38.024954 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:38.024927    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:38.125528 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:38.125465    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:38.226000 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:38.225971    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:38.268834 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.268809    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 18:15:38.269433 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.268963    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 18:15:38.327063 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:38.327031    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:38.343019 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.342984    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 18:10:38 +0000 UTC" deadline="2028-02-25 01:35:10.711522276 +0000 UTC"
Sep 11 18:15:38.343019 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.343015    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12751h19m32.368510652s"
Sep 11 18:15:38.356421 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:38.356395    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb28de82e9dd393a989620953488acea7.slice/crio-6d1cde7bd250a781d89fccec99b31679925ac587f882136ce753f9d735822da8 WatchSource:0}: Error finding container 6d1cde7bd250a781d89fccec99b31679925ac587f882136ce753f9d735822da8: Status 404 returned error can't find the container with id 6d1cde7bd250a781d89fccec99b31679925ac587f882136ce753f9d735822da8
Sep 11 18:15:38.356731 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:38.356715    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode205fce0f56018a4d3daccefe7a004e7.slice/crio-4ba0e50c8b05b028cc6a2cc3e5b3ae603074f8e92930c01b6a14cf51fcd9e077 WatchSource:0}: Error finding container 4ba0e50c8b05b028cc6a2cc3e5b3ae603074f8e92930c01b6a14cf51fcd9e077: Status 404 returned error can't find the container with id 4ba0e50c8b05b028cc6a2cc3e5b3ae603074f8e92930c01b6a14cf51fcd9e077
Sep 11 18:15:38.360154 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.360134    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:15:38.407318 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.407237    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal" event={"ID":"e205fce0f56018a4d3daccefe7a004e7","Type":"ContainerStarted","Data":"4ba0e50c8b05b028cc6a2cc3e5b3ae603074f8e92930c01b6a14cf51fcd9e077"}
Sep 11 18:15:38.408488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.408468    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal" event={"ID":"b28de82e9dd393a989620953488acea7","Type":"ContainerStarted","Data":"6d1cde7bd250a781d89fccec99b31679925ac587f882136ce753f9d735822da8"}
Sep 11 18:15:38.424565 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.424541    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:15:38.427517 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:38.427500    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:38.531400 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:38.531373    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:38.632173 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:38.632147    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:38.738737 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:38.738647    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-82.ec2.internal\" not found"
Sep 11 18:15:38.796551 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.796525    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:15:38.836878 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.836848    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:38.851285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.851263    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 18:15:38.852297 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.852275    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal"
Sep 11 18:15:38.863147 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.863124    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 18:15:38.868747 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:38.868720    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:15:39.315367 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.315331    2578 apiserver.go:52] "Watching apiserver"
Sep 11 18:15:39.324951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.324923    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 18:15:39.326091 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.326059    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-wkkbd","kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal","openshift-dns/node-resolver-v9sp7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal","openshift-multus/multus-x6qdg","openshift-network-operator/iptables-alerter-gqgbt","openshift-ovn-kubernetes/ovnkube-node-hm89p","kube-system/konnectivity-agent-h5km6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk","openshift-cluster-node-tuning-operator/tuned-zkp74","openshift-image-registry/node-ca-d8828","openshift-multus/multus-additional-cni-plugins-64z6w","openshift-multus/network-metrics-daemon-bht8j","openshift-network-diagnostics/network-check-target-czk7w"]
Sep 11 18:15:39.327820 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.327797    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.329799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.329446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:39.329799 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.329535    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:39.330611 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.330223    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:15:39.330611 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.330380    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 18:15:39.330611 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.330447    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gs4jh\""
Sep 11 18:15:39.330863 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.330847    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.332205 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.332182    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:39.332289 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.332244    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:39.333181 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.333161    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 18:15:39.333353 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.333340    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 18:15:39.333509 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.333493    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d4pvq\""
Sep 11 18:15:39.333966 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.333948    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.334737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.334719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.336270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.336248    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.336497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.336252    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 18:15:39.337390 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.337370    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gkf6t\""
Sep 11 18:15:39.337481 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.337409    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 18:15:39.337591 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.337573    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 18:15:39.338282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.337657    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 18:15:39.338282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.337718    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:15:39.338282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.337811    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 18:15:39.338282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.337907    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 18:15:39.338282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.337989    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 18:15:39.338282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.337999    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2vckq\""
Sep 11 18:15:39.338642 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.338622    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 18:15:39.338976 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.338953    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lpfpx\""
Sep 11 18:15:39.339064 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.339024    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 18:15:39.339314 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.339296    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.339474 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.339457    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 18:15:39.341169 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.340786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:15:39.341835 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.341449    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 18:15:39.341932 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.341842    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 18:15:39.342188 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.342167    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:39.342279 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.342233    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:39.346702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.344074    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 18:15:39.346702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.344154    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.346702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.344280    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2s75l\""
Sep 11 18:15:39.346702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.344604    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 18:15:39.346702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.345220    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 18:15:39.346702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.345280    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 18:15:39.346702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.345332    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g5lx4\""
Sep 11 18:15:39.346702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.345672    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 18:15:39.346702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.345843    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 18:15:39.347176 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.346812    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.347432    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h78k6\""
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.347779    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348295    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348342    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-lib-modules\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348381    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a2454c2-8f32-4025-9700-5c3464f05e73-hosts-file\") pod \"node-resolver-v9sp7\" (UID: \"2a2454c2-8f32-4025-9700-5c3464f05e73\") " pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348434    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-cni-netd\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4pgq\" (UniqueName: \"kubernetes.io/projected/89c11824-3186-480a-bb1b-b65c64327900-kube-api-access-j4pgq\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-run-netns\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-hostroot\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348583    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-etc-openvswitch\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-systemd\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-system-cni-dir\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348673    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-multus-socket-dir-parent\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-log-socket\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-kubernetes\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.348871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-sysctl-d\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.348902    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-device-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349038    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e91f825-89cb-4019-b341-699c8c5a902d-ovnkube-script-lib\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2rw2\" (UniqueName: \"kubernetes.io/projected/4e91f825-89cb-4019-b341-699c8c5a902d-kube-api-access-w2rw2\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349142    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-run\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-host\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-run-k8s-cni-cncf-io\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-node-log\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349306    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e91f825-89cb-4019-b341-699c8c5a902d-env-overrides\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/89c11824-3186-480a-bb1b-b65c64327900-etc-tuned\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-systemd-units\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349458    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-run-openvswitch\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-multus-conf-dir\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-slash\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349577    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56921286-0e97-41e5-a110-4c82246dbd9e-multus-daemon-config\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-etc-selinux\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.349699 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv9t6\" (UniqueName: \"kubernetes.io/projected/2a2454c2-8f32-4025-9700-5c3464f05e73-kube-api-access-bv9t6\") pod \"node-resolver-v9sp7\" (UID: \"2a2454c2-8f32-4025-9700-5c3464f05e73\") " pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349716    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-socket-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phkwr\" (UniqueName: \"kubernetes.io/projected/3b505357-e836-4cef-ada2-64371533ae93-kube-api-access-phkwr\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-var-lib-openvswitch\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-sys-fs\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-var-lib-kubelet\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.349994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b79d6\" (UniqueName: \"kubernetes.io/projected/ada13f16-c389-453c-9316-b5db60240062-kube-api-access-b79d6\") pod \"iptables-alerter-gqgbt\" (UID: \"ada13f16-c389-453c-9316-b5db60240062\") " pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.350560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350276    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-run-systemd\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-sysctl-conf\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350635    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a2454c2-8f32-4025-9700-5c3464f05e73-tmp-dir\") pod \"node-resolver-v9sp7\" (UID: \"2a2454c2-8f32-4025-9700-5c3464f05e73\") " pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350668    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-multus-cni-dir\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-var-lib-kubelet\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ada13f16-c389-453c-9316-b5db60240062-iptables-alerter-script\") pod \"iptables-alerter-gqgbt\" (UID: \"ada13f16-c389-453c-9316-b5db60240062\") " pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350820    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-run-ovn\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-cni-bin\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-sys\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89c11824-3186-480a-bb1b-b65c64327900-tmp\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.350994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7vv7\" (UniqueName: \"kubernetes.io/projected/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-kube-api-access-d7vv7\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351022    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56921286-0e97-41e5-a110-4c82246dbd9e-cni-binary-copy\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfbft\" (UniqueName: \"kubernetes.io/projected/56921286-0e97-41e5-a110-4c82246dbd9e-kube-api-access-nfbft\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.351083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ada13f16-c389-453c-9316-b5db60240062-host-slash\") pod \"iptables-alerter-gqgbt\" (UID: \"ada13f16-c389-453c-9316-b5db60240062\") " pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e91f825-89cb-4019-b341-699c8c5a902d-ovnkube-config\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-registration-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351312    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-kubelet\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351348    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-cnibin\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351384    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-var-lib-cni-multus\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-etc-kubernetes\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351504    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-run-netns\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e91f825-89cb-4019-b341-699c8c5a902d-ovn-node-metrics-cert\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351581    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-var-lib-cni-bin\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-run-multus-certs\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-run-ovn-kubernetes\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-modprobe-d\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.351792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.351782    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-sysconfig\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.353995 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.353977    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 18:15:39.354092 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.354007    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4f2fg\""
Sep 11 18:15:39.354261 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.354227    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 18:15:39.363327 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.363297    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 18:15:39.385104 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.385081    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v7dfg"
Sep 11 18:15:39.394931 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.394911    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v7dfg"
Sep 11 18:15:39.437895 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.437871    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 18:15:39.452626 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.452601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56921286-0e97-41e5-a110-4c82246dbd9e-multus-daemon-config\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.452765 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.452639    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-etc-selinux\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.452765 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.452663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv9t6\" (UniqueName: \"kubernetes.io/projected/2a2454c2-8f32-4025-9700-5c3464f05e73-kube-api-access-bv9t6\") pod \"node-resolver-v9sp7\" (UID: \"2a2454c2-8f32-4025-9700-5c3464f05e73\") " pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.452765 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.452679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-socket-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.452765 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.452755    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-etc-selinux\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.453062 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.452711    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phkwr\" (UniqueName: \"kubernetes.io/projected/3b505357-e836-4cef-ada2-64371533ae93-kube-api-access-phkwr\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.453143 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453058    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-socket-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.453143 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-var-lib-openvswitch\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.453143 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.453143 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453121    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-var-lib-openvswitch\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-cnibin\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453165    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-sys-fs\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453206    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-var-lib-kubelet\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453240    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-sys-fs\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453286    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453290    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-var-lib-kubelet\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.453348 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453310    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-cni-binary-copy\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b79d6\" (UniqueName: \"kubernetes.io/projected/ada13f16-c389-453c-9316-b5db60240062-kube-api-access-b79d6\") pod \"iptables-alerter-gqgbt\" (UID: \"ada13f16-c389-453c-9316-b5db60240062\") " pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-run-systemd\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453480    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d5ec55f-0d86-44fc-adb6-b562c0d6f298-host\") pod \"node-ca-d8828\" (UID: \"0d5ec55f-0d86-44fc-adb6-b562c0d6f298\") " pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453515    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-sysctl-conf\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453530    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a2454c2-8f32-4025-9700-5c3464f05e73-tmp-dir\") pod \"node-resolver-v9sp7\" (UID: \"2a2454c2-8f32-4025-9700-5c3464f05e73\") " pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453577    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-multus-cni-dir\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-var-lib-kubelet\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ada13f16-c389-453c-9316-b5db60240062-iptables-alerter-script\") pod \"iptables-alerter-gqgbt\" (UID: \"ada13f16-c389-453c-9316-b5db60240062\") " pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-run-ovn\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453666    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-run-systemd\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-cni-bin\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ffcd016f-d01c-4d6e-8bca-df7a61e26c91-agent-certs\") pod \"konnectivity-agent-h5km6\" (UID: \"ffcd016f-d01c-4d6e-8bca-df7a61e26c91\") " pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453753    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-cni-bin\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.453871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-sys\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89c11824-3186-480a-bb1b-b65c64327900-tmp\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-var-lib-kubelet\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453847    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7vv7\" (UniqueName: \"kubernetes.io/projected/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-kube-api-access-d7vv7\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.453909    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56921286-0e97-41e5-a110-4c82246dbd9e-multus-daemon-config\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454050    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-multus-cni-dir\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454097    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a2454c2-8f32-4025-9700-5c3464f05e73-tmp-dir\") pod \"node-resolver-v9sp7\" (UID: \"2a2454c2-8f32-4025-9700-5c3464f05e73\") " pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-sys\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454107    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-run-ovn\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56921286-0e97-41e5-a110-4c82246dbd9e-cni-binary-copy\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfbft\" (UniqueName: \"kubernetes.io/projected/56921286-0e97-41e5-a110-4c82246dbd9e-kube-api-access-nfbft\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454196    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ada13f16-c389-453c-9316-b5db60240062-host-slash\") pod \"iptables-alerter-gqgbt\" (UID: \"ada13f16-c389-453c-9316-b5db60240062\") " pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e91f825-89cb-4019-b341-699c8c5a902d-ovnkube-config\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454289    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-sysctl-conf\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454305    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ada13f16-c389-453c-9316-b5db60240062-host-slash\") pod \"iptables-alerter-gqgbt\" (UID: \"ada13f16-c389-453c-9316-b5db60240062\") " pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454339    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-registration-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454333    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-kubelet\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.454779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454418    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-cnibin\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-var-lib-cni-multus\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454442    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ada13f16-c389-453c-9316-b5db60240062-iptables-alerter-script\") pod \"iptables-alerter-gqgbt\" (UID: \"ada13f16-c389-453c-9316-b5db60240062\") " pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-kubelet\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-registration-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-cnibin\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-var-lib-cni-multus\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-etc-kubernetes\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-run-netns\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e91f825-89cb-4019-b341-699c8c5a902d-ovn-node-metrics-cert\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-etc-kubernetes\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454622    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-system-cni-dir\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8pl8\" (UniqueName: \"kubernetes.io/projected/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-kube-api-access-b8pl8\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-run-netns\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454697    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-var-lib-cni-bin\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-run-multus-certs\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-run-ovn-kubernetes\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.455712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-modprobe-d\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454812    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-sysconfig\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454833    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-var-lib-cni-bin\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-lib-modules\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454887    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a2454c2-8f32-4025-9700-5c3464f05e73-hosts-file\") pod \"node-resolver-v9sp7\" (UID: \"2a2454c2-8f32-4025-9700-5c3464f05e73\") " pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-cni-netd\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454924    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e91f825-89cb-4019-b341-699c8c5a902d-ovnkube-config\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454941    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-lib-modules\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4pgq\" (UniqueName: \"kubernetes.io/projected/89c11824-3186-480a-bb1b-b65c64327900-kube-api-access-j4pgq\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.454985    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-run-multus-certs\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-modprobe-d\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455046    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a2454c2-8f32-4025-9700-5c3464f05e73-hosts-file\") pod \"node-resolver-v9sp7\" (UID: \"2a2454c2-8f32-4025-9700-5c3464f05e73\") " pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455011    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-sysconfig\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-cni-netd\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-run-netns\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455168    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-run-ovn-kubernetes\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455173    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-run-netns\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.456592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-hostroot\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.455235    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-etc-openvswitch\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455266    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-systemd\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-system-cni-dir\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.455317    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs podName:c3d9bcff-2714-44ac-8bd8-e288668a8a7b nodeName:}" failed. No retries permitted until 2026-09-11 18:15:39.95529292 +0000 UTC m=+2.990509584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs") pod "network-metrics-daemon-bht8j" (UID: "c3d9bcff-2714-44ac-8bd8-e288668a8a7b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455324    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-etc-openvswitch\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-hostroot\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-multus-socket-dir-parent\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455383    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-systemd\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-log-socket\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455402    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-multus-socket-dir-parent\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455382    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-system-cni-dir\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455409    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-log-socket\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-kubernetes\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455458    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-sysctl-d\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455472    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-kubernetes\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.457402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-device-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e91f825-89cb-4019-b341-699c8c5a902d-ovnkube-script-lib\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56921286-0e97-41e5-a110-4c82246dbd9e-cni-binary-copy\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2rw2\" (UniqueName: \"kubernetes.io/projected/4e91f825-89cb-4019-b341-699c8c5a902d-kube-api-access-w2rw2\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6a6009bd-7d00-437f-9cf4-9161d79ab722-dbus\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455581    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-etc-sysctl-d\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455581    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3b505357-e836-4cef-ada2-64371533ae93-device-dir\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-run\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455640    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ffcd016f-d01c-4d6e-8bca-df7a61e26c91-konnectivity-ca\") pod \"konnectivity-agent-h5km6\" (UID: \"ffcd016f-d01c-4d6e-8bca-df7a61e26c91\") " pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6a6009bd-7d00-437f-9cf4-9161d79ab722-kubelet-config\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455686    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d5ec55f-0d86-44fc-adb6-b562c0d6f298-serviceca\") pod \"node-ca-d8828\" (UID: \"0d5ec55f-0d86-44fc-adb6-b562c0d6f298\") " pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455726    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-run\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455747    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-host\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-run-k8s-cni-cncf-io\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455842    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/89c11824-3186-480a-bb1b-b65c64327900-host\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-node-log\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.458253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455885    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e91f825-89cb-4019-b341-699c8c5a902d-env-overrides\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455900    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-host-run-k8s-cni-cncf-io\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/89c11824-3186-480a-bb1b-b65c64327900-etc-tuned\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-systemd-units\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-run-openvswitch\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455966    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-node-log\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.455989    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np7mp\" (UniqueName: \"kubernetes.io/projected/0d5ec55f-0d86-44fc-adb6-b562c0d6f298-kube-api-access-np7mp\") pod \"node-ca-d8828\" (UID: \"0d5ec55f-0d86-44fc-adb6-b562c0d6f298\") " pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.456014    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-multus-conf-dir\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.456038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-slash\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.456053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-systemd-units\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.456118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-host-slash\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.456105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e91f825-89cb-4019-b341-699c8c5a902d-ovnkube-script-lib\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.456168    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e91f825-89cb-4019-b341-699c8c5a902d-run-openvswitch\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.456221    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56921286-0e97-41e5-a110-4c82246dbd9e-multus-conf-dir\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.456288    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e91f825-89cb-4019-b341-699c8c5a902d-env-overrides\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.458878    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e91f825-89cb-4019-b341-699c8c5a902d-ovn-node-metrics-cert\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.458883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/89c11824-3186-480a-bb1b-b65c64327900-etc-tuned\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.459832 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.459042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89c11824-3186-480a-bb1b-b65c64327900-tmp\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.459832 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.459690    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:15:39.459832 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.459712    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:15:39.459832 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.459726    2578 projected.go:196] Error preparing data for projected volume kube-api-access-q468p for pod openshift-network-diagnostics/network-check-target-czk7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:39.459832 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.459799    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p podName:75588279-fbb8-4a19-a0c2-6ea50f680c22 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:39.959781372 +0000 UTC m=+2.994998025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q468p" (UniqueName: "kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p") pod "network-check-target-czk7w" (UID: "75588279-fbb8-4a19-a0c2-6ea50f680c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:39.462037 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.462016    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv9t6\" (UniqueName: \"kubernetes.io/projected/2a2454c2-8f32-4025-9700-5c3464f05e73-kube-api-access-bv9t6\") pod \"node-resolver-v9sp7\" (UID: \"2a2454c2-8f32-4025-9700-5c3464f05e73\") " pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.462260 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.462234    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b79d6\" (UniqueName: \"kubernetes.io/projected/ada13f16-c389-453c-9316-b5db60240062-kube-api-access-b79d6\") pod \"iptables-alerter-gqgbt\" (UID: \"ada13f16-c389-453c-9316-b5db60240062\") " pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.462674 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.462638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phkwr\" (UniqueName: \"kubernetes.io/projected/3b505357-e836-4cef-ada2-64371533ae93-kube-api-access-phkwr\") pod \"aws-ebs-csi-driver-node-q7wzk\" (UID: \"3b505357-e836-4cef-ada2-64371533ae93\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.464160 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.464141    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfbft\" (UniqueName: \"kubernetes.io/projected/56921286-0e97-41e5-a110-4c82246dbd9e-kube-api-access-nfbft\") pod \"multus-x6qdg\" (UID: \"56921286-0e97-41e5-a110-4c82246dbd9e\") " pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.464606 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.464586    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7vv7\" (UniqueName: \"kubernetes.io/projected/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-kube-api-access-d7vv7\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:39.465454 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.465431    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2rw2\" (UniqueName: \"kubernetes.io/projected/4e91f825-89cb-4019-b341-699c8c5a902d-kube-api-access-w2rw2\") pod \"ovnkube-node-hm89p\" (UID: \"4e91f825-89cb-4019-b341-699c8c5a902d\") " pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.466380 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.466361    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4pgq\" (UniqueName: \"kubernetes.io/projected/89c11824-3186-480a-bb1b-b65c64327900-kube-api-access-j4pgq\") pod \"tuned-zkp74\" (UID: \"89c11824-3186-480a-bb1b-b65c64327900\") " pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.556461 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-cnibin\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.556461 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556456    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:39.556461 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556462    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-cnibin\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.556798 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-cni-binary-copy\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.556798 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.556798 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.556798 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d5ec55f-0d86-44fc-adb6-b562c0d6f298-host\") pod \"node-ca-d8828\" (UID: \"0d5ec55f-0d86-44fc-adb6-b562c0d6f298\") " pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.556798 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.556575    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:39.556798 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.556660    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret podName:6a6009bd-7d00-437f-9cf4-9161d79ab722 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:40.056637494 +0000 UTC m=+3.091854161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret") pod "global-pull-secret-syncer-wkkbd" (UID: "6a6009bd-7d00-437f-9cf4-9161d79ab722") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:39.557110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ffcd016f-d01c-4d6e-8bca-df7a61e26c91-agent-certs\") pod \"konnectivity-agent-h5km6\" (UID: \"ffcd016f-d01c-4d6e-8bca-df7a61e26c91\") " pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:15:39.557110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-system-cni-dir\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.557110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.556936    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8pl8\" (UniqueName: \"kubernetes.io/projected/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-kube-api-access-b8pl8\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.557110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6a6009bd-7d00-437f-9cf4-9161d79ab722-dbus\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:39.557110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557033    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.557110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ffcd016f-d01c-4d6e-8bca-df7a61e26c91-konnectivity-ca\") pod \"konnectivity-agent-h5km6\" (UID: \"ffcd016f-d01c-4d6e-8bca-df7a61e26c91\") " pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:15:39.557110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557088    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6a6009bd-7d00-437f-9cf4-9161d79ab722-kubelet-config\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:39.557110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d5ec55f-0d86-44fc-adb6-b562c0d6f298-serviceca\") pod \"node-ca-d8828\" (UID: \"0d5ec55f-0d86-44fc-adb6-b562c0d6f298\") " pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.557498 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np7mp\" (UniqueName: \"kubernetes.io/projected/0d5ec55f-0d86-44fc-adb6-b562c0d6f298-kube-api-access-np7mp\") pod \"node-ca-d8828\" (UID: \"0d5ec55f-0d86-44fc-adb6-b562c0d6f298\") " pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.557498 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.557498 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-cni-binary-copy\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.557498 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557300    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6a6009bd-7d00-437f-9cf4-9161d79ab722-dbus\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:39.557498 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d5ec55f-0d86-44fc-adb6-b562c0d6f298-host\") pod \"node-ca-d8828\" (UID: \"0d5ec55f-0d86-44fc-adb6-b562c0d6f298\") " pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.557498 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557355    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-system-cni-dir\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.557498 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557360    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.557498 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6a6009bd-7d00-437f-9cf4-9161d79ab722-kubelet-config\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:39.557929 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557826    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d5ec55f-0d86-44fc-adb6-b562c0d6f298-serviceca\") pod \"node-ca-d8828\" (UID: \"0d5ec55f-0d86-44fc-adb6-b562c0d6f298\") " pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.557929 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.557913    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.558046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.558007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ffcd016f-d01c-4d6e-8bca-df7a61e26c91-konnectivity-ca\") pod \"konnectivity-agent-h5km6\" (UID: \"ffcd016f-d01c-4d6e-8bca-df7a61e26c91\") " pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:15:39.559138 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.559118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ffcd016f-d01c-4d6e-8bca-df7a61e26c91-agent-certs\") pod \"konnectivity-agent-h5km6\" (UID: \"ffcd016f-d01c-4d6e-8bca-df7a61e26c91\") " pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:15:39.566294 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.566243    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np7mp\" (UniqueName: \"kubernetes.io/projected/0d5ec55f-0d86-44fc-adb6-b562c0d6f298-kube-api-access-np7mp\") pod \"node-ca-d8828\" (UID: \"0d5ec55f-0d86-44fc-adb6-b562c0d6f298\") " pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.566392 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.566315    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8pl8\" (UniqueName: \"kubernetes.io/projected/0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd-kube-api-access-b8pl8\") pod \"multus-additional-cni-plugins-64z6w\" (UID: \"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd\") " pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.642453 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.642407    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zkp74"
Sep 11 18:15:39.652865 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.652843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v9sp7"
Sep 11 18:15:39.663240 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.663213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x6qdg"
Sep 11 18:15:39.669008 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.668993    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gqgbt"
Sep 11 18:15:39.675877 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.675849    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk"
Sep 11 18:15:39.683754 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.683732    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:15:39.688866 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.688845    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:15:39.696838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.696818    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d8828"
Sep 11 18:15:39.701849 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.701827    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-64z6w"
Sep 11 18:15:39.866938 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.866858    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:15:39.961313 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.961273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:39.961501 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:39.961349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:39.961501 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.961443    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:39.961501 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.961460    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:15:39.961501 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.961479    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:15:39.961501 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.961493    2578 projected.go:196] Error preparing data for projected volume kube-api-access-q468p for pod openshift-network-diagnostics/network-check-target-czk7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:39.961501 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.961509    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs podName:c3d9bcff-2714-44ac-8bd8-e288668a8a7b nodeName:}" failed. No retries permitted until 2026-09-11 18:15:40.961489224 +0000 UTC m=+3.996705892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs") pod "network-metrics-daemon-bht8j" (UID: "c3d9bcff-2714-44ac-8bd8-e288668a8a7b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:39.961809 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:39.961530    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p podName:75588279-fbb8-4a19-a0c2-6ea50f680c22 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:40.961519857 +0000 UTC m=+3.996736512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q468p" (UniqueName: "kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p") pod "network-check-target-czk7w" (UID: "75588279-fbb8-4a19-a0c2-6ea50f680c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:40.062445 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:40.062413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:40.062611 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:40.062591    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:40.062689 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:40.062675    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret podName:6a6009bd-7d00-437f-9cf4-9161d79ab722 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:41.062656145 +0000 UTC m=+4.097872795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret") pod "global-pull-secret-syncer-wkkbd" (UID: "6a6009bd-7d00-437f-9cf4-9161d79ab722") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:40.394002 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:40.393909    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b1b8ab8_1bd7_4ebe_b274_9a9689afc6cd.slice/crio-942f04871ddb17845b92529d92ee55ae00ac5facf486de2b33feeb863fb40a02 WatchSource:0}: Error finding container 942f04871ddb17845b92529d92ee55ae00ac5facf486de2b33feeb863fb40a02: Status 404 returned error can't find the container with id 942f04871ddb17845b92529d92ee55ae00ac5facf486de2b33feeb863fb40a02
Sep 11 18:15:40.397393 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:40.397364    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 18:10:40 +0000 UTC" deadline="2028-03-26 08:23:16.505639887 +0000 UTC"
Sep 11 18:15:40.397393 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:40.397392    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13478h7m36.108251603s"
Sep 11 18:15:40.404372 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:40.404354    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:40.404464 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:40.404447    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:40.407468 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:40.407402    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2454c2_8f32_4025_9700_5c3464f05e73.slice/crio-81a3a934dc0fb21eba58be70ace63190acd5c52fbbe478b0ad169e65a70d894e WatchSource:0}: Error finding container 81a3a934dc0fb21eba58be70ace63190acd5c52fbbe478b0ad169e65a70d894e: Status 404 returned error can't find the container with id 81a3a934dc0fb21eba58be70ace63190acd5c52fbbe478b0ad169e65a70d894e
Sep 11 18:15:40.411169 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:40.410413    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c11824_3186_480a_bb1b_b65c64327900.slice/crio-d3403962d467bc09a45a489615461a0de0581f83ffcfd8a9d9670969e1b907f8 WatchSource:0}: Error finding container d3403962d467bc09a45a489615461a0de0581f83ffcfd8a9d9670969e1b907f8: Status 404 returned error can't find the container with id d3403962d467bc09a45a489615461a0de0581f83ffcfd8a9d9670969e1b907f8
Sep 11 18:15:40.411391 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:40.411266    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5ec55f_0d86_44fc_adb6_b562c0d6f298.slice/crio-a18fe4d96267637e87eda1030c543e09285034324b89df8d4a23a88fcdeaa807 WatchSource:0}: Error finding container a18fe4d96267637e87eda1030c543e09285034324b89df8d4a23a88fcdeaa807: Status 404 returned error can't find the container with id a18fe4d96267637e87eda1030c543e09285034324b89df8d4a23a88fcdeaa807
Sep 11 18:15:40.413321 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:40.413144    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v9sp7" event={"ID":"2a2454c2-8f32-4025-9700-5c3464f05e73","Type":"ContainerStarted","Data":"81a3a934dc0fb21eba58be70ace63190acd5c52fbbe478b0ad169e65a70d894e"}
Sep 11 18:15:40.414056 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:40.413785    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b505357_e836_4cef_ada2_64371533ae93.slice/crio-ad65cde2b0180202a66cdf4ad1553c738c9b37b1f3b5b1d5dd6787c6bf3f370d WatchSource:0}: Error finding container ad65cde2b0180202a66cdf4ad1553c738c9b37b1f3b5b1d5dd6787c6bf3f370d: Status 404 returned error can't find the container with id ad65cde2b0180202a66cdf4ad1553c738c9b37b1f3b5b1d5dd6787c6bf3f370d
Sep 11 18:15:40.415059 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:40.415035    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada13f16_c389_453c_9316_b5db60240062.slice/crio-e987d64186f72ad147f2aebda08a3e36f8c8a82280c7daacd6c6dc1c71545c42 WatchSource:0}: Error finding container e987d64186f72ad147f2aebda08a3e36f8c8a82280c7daacd6c6dc1c71545c42: Status 404 returned error can't find the container with id e987d64186f72ad147f2aebda08a3e36f8c8a82280c7daacd6c6dc1c71545c42
Sep 11 18:15:40.415323 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:15:40.415295    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e91f825_89cb_4019_b341_699c8c5a902d.slice/crio-69049f0768be1fb8d55bb0e9be9cce95c99233f4b19bfcd48fc50b2bd6b13a40 WatchSource:0}: Error finding container 69049f0768be1fb8d55bb0e9be9cce95c99233f4b19bfcd48fc50b2bd6b13a40: Status 404 returned error can't find the container with id 69049f0768be1fb8d55bb0e9be9cce95c99233f4b19bfcd48fc50b2bd6b13a40
Sep 11 18:15:40.416132 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:40.416001    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-64z6w" event={"ID":"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd","Type":"ContainerStarted","Data":"942f04871ddb17845b92529d92ee55ae00ac5facf486de2b33feeb863fb40a02"}
Sep 11 18:15:40.969108 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:40.968766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:40.969291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:40.969138    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:40.969291 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:40.968922    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:15:40.969413 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:40.969320    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:15:40.969413 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:40.969337    2578 projected.go:196] Error preparing data for projected volume kube-api-access-q468p for pod openshift-network-diagnostics/network-check-target-czk7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:40.969413 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:40.969403    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p podName:75588279-fbb8-4a19-a0c2-6ea50f680c22 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:42.969382896 +0000 UTC m=+6.004599568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q468p" (UniqueName: "kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p") pod "network-check-target-czk7w" (UID: "75588279-fbb8-4a19-a0c2-6ea50f680c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:40.969940 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:40.969265    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:40.969940 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:40.969835    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs podName:c3d9bcff-2714-44ac-8bd8-e288668a8a7b nodeName:}" failed. No retries permitted until 2026-09-11 18:15:42.969816234 +0000 UTC m=+6.005032887 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs") pod "network-metrics-daemon-bht8j" (UID: "c3d9bcff-2714-44ac-8bd8-e288668a8a7b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:41.073250 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.069534    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:41.073250 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:41.069699    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:41.073250 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:41.069792    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret podName:6a6009bd-7d00-437f-9cf4-9161d79ab722 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:43.069770203 +0000 UTC m=+6.104986878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret") pod "global-pull-secret-syncer-wkkbd" (UID: "6a6009bd-7d00-437f-9cf4-9161d79ab722") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:41.397734 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.397562    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 18:10:40 +0000 UTC" deadline="2028-05-09 01:52:56.764566613 +0000 UTC"
Sep 11 18:15:41.397734 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.397629    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14527h37m15.366942366s"
Sep 11 18:15:41.404872 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.404343    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:41.404872 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:41.404474    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:41.405372 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.405204    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:41.405372 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:41.405311    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:41.440789 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.440122    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal" event={"ID":"e205fce0f56018a4d3daccefe7a004e7","Type":"ContainerStarted","Data":"8cb8d1088b4d06c581f2f8ff23ee78052a56c8d4ba48317464a51789ce25cc23"}
Sep 11 18:15:41.444843 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.444778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x6qdg" event={"ID":"56921286-0e97-41e5-a110-4c82246dbd9e","Type":"ContainerStarted","Data":"39c5f8f876f05d411b366f8e0fef5545ca230f3ddc7c0a350f4999cd68a11b1d"}
Sep 11 18:15:41.455769 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.455664    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerStarted","Data":"69049f0768be1fb8d55bb0e9be9cce95c99233f4b19bfcd48fc50b2bd6b13a40"}
Sep 11 18:15:41.471164 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.471133    2578 generic.go:356] "Generic (PLEG): container finished" podID="b28de82e9dd393a989620953488acea7" containerID="a2c9651bf81ffde7264573664d5e8683c1321818c93f34501f74135192819155" exitCode=0
Sep 11 18:15:41.471388 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.471364    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal" event={"ID":"b28de82e9dd393a989620953488acea7","Type":"ContainerDied","Data":"a2c9651bf81ffde7264573664d5e8683c1321818c93f34501f74135192819155"}
Sep 11 18:15:41.482089 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.482060    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h5km6" event={"ID":"ffcd016f-d01c-4d6e-8bca-df7a61e26c91","Type":"ContainerStarted","Data":"9026d4ea88a03ff1a52bcaa91792e39638cc7e6654df42620404be3ed41e7ba2"}
Sep 11 18:15:41.486783 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.486712    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-82.ec2.internal" podStartSLOduration=2.486693817 podStartE2EDuration="2.486693817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:15:41.455634956 +0000 UTC m=+4.490851645" watchObservedRunningTime="2026-09-11 18:15:41.486693817 +0000 UTC m=+4.521910492"
Sep 11 18:15:41.497819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.497785    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gqgbt" event={"ID":"ada13f16-c389-453c-9316-b5db60240062","Type":"ContainerStarted","Data":"e987d64186f72ad147f2aebda08a3e36f8c8a82280c7daacd6c6dc1c71545c42"}
Sep 11 18:15:41.510098 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.510070    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk" event={"ID":"3b505357-e836-4cef-ada2-64371533ae93","Type":"ContainerStarted","Data":"ad65cde2b0180202a66cdf4ad1553c738c9b37b1f3b5b1d5dd6787c6bf3f370d"}
Sep 11 18:15:41.516199 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.516162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d8828" event={"ID":"0d5ec55f-0d86-44fc-adb6-b562c0d6f298","Type":"ContainerStarted","Data":"a18fe4d96267637e87eda1030c543e09285034324b89df8d4a23a88fcdeaa807"}
Sep 11 18:15:41.521035 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:41.521005    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zkp74" event={"ID":"89c11824-3186-480a-bb1b-b65c64327900","Type":"ContainerStarted","Data":"d3403962d467bc09a45a489615461a0de0581f83ffcfd8a9d9670969e1b907f8"}
Sep 11 18:15:42.404640 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:42.404105    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:42.404640 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:42.404256    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:42.537827 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:42.532156    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal" event={"ID":"b28de82e9dd393a989620953488acea7","Type":"ContainerStarted","Data":"07efbb3dae360ede27d782e0cba4b5242f9d6e7ff617a6abc3ec70e3ecd0c1e9"}
Sep 11 18:15:42.987695 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:42.986872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:42.987695 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:42.986962    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:42.987695 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:42.987117    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:42.987695 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:42.987182    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs podName:c3d9bcff-2714-44ac-8bd8-e288668a8a7b nodeName:}" failed. No retries permitted until 2026-09-11 18:15:46.987162259 +0000 UTC m=+10.022378913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs") pod "network-metrics-daemon-bht8j" (UID: "c3d9bcff-2714-44ac-8bd8-e288668a8a7b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:42.987695 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:42.987277    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:15:42.987695 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:42.987292    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:15:42.987695 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:42.987305    2578 projected.go:196] Error preparing data for projected volume kube-api-access-q468p for pod openshift-network-diagnostics/network-check-target-czk7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:42.987695 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:42.987342    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p podName:75588279-fbb8-4a19-a0c2-6ea50f680c22 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:46.987329637 +0000 UTC m=+10.022546302 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q468p" (UniqueName: "kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p") pod "network-check-target-czk7w" (UID: "75588279-fbb8-4a19-a0c2-6ea50f680c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:43.088457 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:43.088415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:43.088705 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:43.088682    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:43.088803 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:43.088789    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret podName:6a6009bd-7d00-437f-9cf4-9161d79ab722 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:47.088766268 +0000 UTC m=+10.123982921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret") pod "global-pull-secret-syncer-wkkbd" (UID: "6a6009bd-7d00-437f-9cf4-9161d79ab722") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:43.404424 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:43.404339    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:43.404584 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:43.404483    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:43.404900 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:43.404845    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:43.405472 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:43.404983    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:44.404248 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:44.404210    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:44.404432 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:44.404368    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:45.404619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:45.404138    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:45.404619 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:45.404268    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:45.404619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:45.404543    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:45.405234 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:45.404630    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:46.404434 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:46.403915    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:46.404434 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:46.404049    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:47.019985 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:47.019944    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:47.020492 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:47.020035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:47.020492 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.020201    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:15:47.020492 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.020223    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:15:47.020492 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.020236    2578 projected.go:196] Error preparing data for projected volume kube-api-access-q468p for pod openshift-network-diagnostics/network-check-target-czk7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:47.020492 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.020293    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p podName:75588279-fbb8-4a19-a0c2-6ea50f680c22 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:55.020273874 +0000 UTC m=+18.055490527 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q468p" (UniqueName: "kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p") pod "network-check-target-czk7w" (UID: "75588279-fbb8-4a19-a0c2-6ea50f680c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:47.020868 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.020712    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:47.020868 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.020785    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs podName:c3d9bcff-2714-44ac-8bd8-e288668a8a7b nodeName:}" failed. No retries permitted until 2026-09-11 18:15:55.020766868 +0000 UTC m=+18.055983538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs") pod "network-metrics-daemon-bht8j" (UID: "c3d9bcff-2714-44ac-8bd8-e288668a8a7b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:47.121163 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:47.121123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:47.121352 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.121332    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:47.121415 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.121402    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret podName:6a6009bd-7d00-437f-9cf4-9161d79ab722 nodeName:}" failed. No retries permitted until 2026-09-11 18:15:55.121380491 +0000 UTC m=+18.156597162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret") pod "global-pull-secret-syncer-wkkbd" (UID: "6a6009bd-7d00-437f-9cf4-9161d79ab722") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:47.404931 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:47.404852    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:47.405232 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.405205    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:47.409220 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:47.409024    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:47.409220 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:47.409171    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:48.404140 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:48.404102    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:48.404800 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:48.404254    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:49.406528 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:49.406489    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:49.406984 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:49.406626    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:49.406984 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:49.406689    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:49.406984 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:49.406779    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:50.403804 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:50.403772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:50.403986 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:50.403919    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:51.403631 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:51.403596    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:51.404101 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:51.403598    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:51.404101 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:51.403717    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:51.404101 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:51.403791    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:52.403736 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:52.403703    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:52.404149 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:52.403839    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:53.404064 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:53.404024    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:53.404064 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:53.404031    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:53.404497 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:53.404151    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:53.404497 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:53.404293    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:54.404404 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:54.404368    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:54.404926 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:54.404500    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:55.084397 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:55.084366    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:55.084560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:55.084412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:55.084560 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.084517    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:55.084630 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.084580    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs podName:c3d9bcff-2714-44ac-8bd8-e288668a8a7b nodeName:}" failed. No retries permitted until 2026-09-11 18:16:11.084565555 +0000 UTC m=+34.119782218 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs") pod "network-metrics-daemon-bht8j" (UID: "c3d9bcff-2714-44ac-8bd8-e288668a8a7b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:15:55.084630 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.084522    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:15:55.084630 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.084616    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:15:55.084630 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.084628    2578 projected.go:196] Error preparing data for projected volume kube-api-access-q468p for pod openshift-network-diagnostics/network-check-target-czk7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:55.084767 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.084669    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p podName:75588279-fbb8-4a19-a0c2-6ea50f680c22 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:11.084658491 +0000 UTC m=+34.119875140 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q468p" (UniqueName: "kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p") pod "network-check-target-czk7w" (UID: "75588279-fbb8-4a19-a0c2-6ea50f680c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:15:55.184738 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:55.184696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:55.184900 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.184881    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:55.184976 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.184964    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret podName:6a6009bd-7d00-437f-9cf4-9161d79ab722 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:11.184942476 +0000 UTC m=+34.220159137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret") pod "global-pull-secret-syncer-wkkbd" (UID: "6a6009bd-7d00-437f-9cf4-9161d79ab722") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:15:55.404084 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:55.403998    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:55.404233 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.404131    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:55.404233 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:55.404159    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:55.404353 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:55.404267    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:56.404162 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:56.404132    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:56.404805 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:56.404243    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:57.404251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:57.404225    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:57.404725 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:57.404336    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:57.404725 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:57.404447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:57.404725 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:57.404571    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:58.403618 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:58.403586    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:15:58.403778 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:58.403741    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:15:59.403987 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.403737    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:15:59.404278 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:59.404057    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:15:59.404278 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.404092    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:15:59.404278 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:15:59.404205    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:15:59.584594 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.584555    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h5km6" event={"ID":"ffcd016f-d01c-4d6e-8bca-df7a61e26c91","Type":"ContainerStarted","Data":"71dab20a829c995ac937eb5a3355c56d9690da1fccb207b02e89d06068311dfd"}
Sep 11 18:15:59.587099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.586331    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk" event={"ID":"3b505357-e836-4cef-ada2-64371533ae93","Type":"ContainerStarted","Data":"0de8358defbe1c8c605f7f65ad23c1123eb2d6192e125b97d6ad8270cb8107a9"}
Sep 11 18:15:59.588260 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.588209    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d8828" event={"ID":"0d5ec55f-0d86-44fc-adb6-b562c0d6f298","Type":"ContainerStarted","Data":"af0b2e04528d54cb99c6ce213daee418c53a0ebe4cea6475e1549a9baf3eeb1e"}
Sep 11 18:15:59.589969 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.589949    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zkp74" event={"ID":"89c11824-3186-480a-bb1b-b65c64327900","Type":"ContainerStarted","Data":"2f9cfdc2057f2491ad7f22e38c55067b89db511247f61f458946aba25fe49b69"}
Sep 11 18:15:59.592884 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.592297    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:15:59.592884 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.592638    2578 generic.go:356] "Generic (PLEG): container finished" podID="4e91f825-89cb-4019-b341-699c8c5a902d" containerID="343c0f4bab7d82513eaf2c909da67222ae9a41f3e2df92076f0480238e138e36" exitCode=1
Sep 11 18:15:59.592884 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.592704    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerDied","Data":"343c0f4bab7d82513eaf2c909da67222ae9a41f3e2df92076f0480238e138e36"}
Sep 11 18:15:59.592884 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.592774    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerStarted","Data":"bb1aa76b40b41671dc198f15603af1158a5f9285d7034ec2c1b782adaae0ab1a"}
Sep 11 18:15:59.594268 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.594246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v9sp7" event={"ID":"2a2454c2-8f32-4025-9700-5c3464f05e73","Type":"ContainerStarted","Data":"d9c02ee37a812c35dac44df6ce8792fd2cea8e4de856135fae22e5444b752248"}
Sep 11 18:15:59.597415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.597395    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-64z6w" event={"ID":"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd","Type":"ContainerStarted","Data":"a515ed4d6d6c19b1d0b511065116d1439433636bf06c4a436bc3896c86dd1e53"}
Sep 11 18:15:59.603193 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.603142    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-82.ec2.internal" podStartSLOduration=20.603125662 podStartE2EDuration="20.603125662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:15:42.549014167 +0000 UTC m=+5.584230842" watchObservedRunningTime="2026-09-11 18:15:59.603125662 +0000 UTC m=+22.638342335"
Sep 11 18:15:59.603506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.603478    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h5km6" podStartSLOduration=2.867557813 podStartE2EDuration="21.603470136s" totalImagesPullingTime="18.735912323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:15:59.602952241 +0000 UTC m=+22.638168913" watchObservedRunningTime="2026-09-11 18:15:59.603470136 +0000 UTC m=+22.638686805"
Sep 11 18:15:59.618594 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.618557    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d8828" podStartSLOduration=2.845418118 podStartE2EDuration="21.618540862s" totalImagesPullingTime="18.773122744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:15:59.618265763 +0000 UTC m=+22.653482433" watchObservedRunningTime="2026-09-11 18:15:59.618540862 +0000 UTC m=+22.653757533"
Sep 11 18:15:59.635153 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.635118    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zkp74" podStartSLOduration=2.882561227 podStartE2EDuration="21.635105533s" totalImagesPullingTime="18.752544306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:15:59.634940536 +0000 UTC m=+22.670157203" watchObservedRunningTime="2026-09-11 18:15:59.635105533 +0000 UTC m=+22.670322203"
Sep 11 18:15:59.673700 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:15:59.673497    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-v9sp7" podStartSLOduration=2.895611031 podStartE2EDuration="21.673480722s" totalImagesPullingTime="18.777869691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:15:59.672918015 +0000 UTC m=+22.708134686" watchObservedRunningTime="2026-09-11 18:15:59.673480722 +0000 UTC m=+22.708697391"
Sep 11 18:16:00.404088 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.404053    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:00.404640 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:00.404191    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:16:00.601838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.601813    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:16:00.602252 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.602230    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerStarted","Data":"c197bb1b47f131163f1286e085e4e5356a27db9e7117d25e658f6cc53c9bace3"}
Sep 11 18:16:00.602318 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.602303    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerStarted","Data":"c092668714bca19f48e942fa4f0d1c896d600f33261beb6c9b73c3c6ba08e53f"}
Sep 11 18:16:00.602367 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.602351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerStarted","Data":"9997306ec966e6af65a390863e15062a65c3f924b524ea2105e09c147110bc71"}
Sep 11 18:16:00.602403 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.602388    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerStarted","Data":"5809f91aac57383fe2d73c30da9276e446cab0fd306b5de845298b1f86fb548b"}
Sep 11 18:16:00.603659 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.603635    2578 generic.go:356] "Generic (PLEG): container finished" podID="0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd" containerID="a515ed4d6d6c19b1d0b511065116d1439433636bf06c4a436bc3896c86dd1e53" exitCode=0
Sep 11 18:16:00.603807 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.603720    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-64z6w" event={"ID":"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd","Type":"ContainerDied","Data":"a515ed4d6d6c19b1d0b511065116d1439433636bf06c4a436bc3896c86dd1e53"}
Sep 11 18:16:00.605124 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.605103    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x6qdg" event={"ID":"56921286-0e97-41e5-a110-4c82246dbd9e","Type":"ContainerStarted","Data":"c49f679277ad9d46adc89fb7e0d83d98d4596b5eda1af77f16d446d063b21387"}
Sep 11 18:16:00.645615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.645577    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x6qdg" podStartSLOduration=3.49639809 podStartE2EDuration="22.645564567s" totalImagesPullingTime="19.149166477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:00.645529002 +0000 UTC m=+23.680745671" watchObservedRunningTime="2026-09-11 18:16:00.645564567 +0000 UTC m=+23.680781237"
Sep 11 18:16:00.746075 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:00.746051    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 18:16:01.398073 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.397963    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T18:16:00.74606839Z","UUID":"dc8a1652-caf9-40a0-8c9e-654b6956ad24","Handler":null,"Name":"","Endpoint":""}
Sep 11 18:16:01.399692 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.399655    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 18:16:01.399815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.399711    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 18:16:01.404053 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.403599    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:01.404053 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.403616    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:01.404053 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:01.403737    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:16:01.404053 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:01.403875    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:16:01.610216 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.609741    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gqgbt" event={"ID":"ada13f16-c389-453c-9316-b5db60240062","Type":"ContainerStarted","Data":"b64b11078e328fa575b5e05413bbbdf6062c06e5efbc1d5c5b8802021ee64cb5"}
Sep 11 18:16:01.612607 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.612578    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk" event={"ID":"3b505357-e836-4cef-ada2-64371533ae93","Type":"ContainerStarted","Data":"a74aa16238f53e05d35249505794c085975324c25d5857e68976611a02358fe9"}
Sep 11 18:16:01.625077 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.625021    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gqgbt" podStartSLOduration=4.84838973 podStartE2EDuration="23.625005465s" totalImagesPullingTime="18.776615735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:01.624457825 +0000 UTC m=+24.659674494" watchObservedRunningTime="2026-09-11 18:16:01.625005465 +0000 UTC m=+24.660222338"
Sep 11 18:16:01.964096 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.964066    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:16:01.964885 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:01.964867    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:16:02.404417 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:02.404388    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:02.404865 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:02.404490    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:16:02.618084 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:02.618049    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk" event={"ID":"3b505357-e836-4cef-ada2-64371533ae93","Type":"ContainerStarted","Data":"a43df7e60875681e36d97f126f2f3a871f64c093e4ad8eac4c94df823a6c6dc3"}
Sep 11 18:16:02.621618 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:02.621525    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:16:02.622024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:02.621999    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerStarted","Data":"4ebdbc1f1e5f5a1966f470dccf924c2a3be5d42c15dc89447124c976910db673"}
Sep 11 18:16:02.622386 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:02.622361    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:16:02.622890 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:02.622876    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h5km6"
Sep 11 18:16:02.635485 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:02.635436    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q7wzk" podStartSLOduration=3.557700246 podStartE2EDuration="24.635420024s" totalImagesPullingTime="21.077719778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:02.635142155 +0000 UTC m=+25.670358815" watchObservedRunningTime="2026-09-11 18:16:02.635420024 +0000 UTC m=+25.670636695"
Sep 11 18:16:03.404544 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:03.404509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:03.405176 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:03.404630    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:16:03.405176 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:03.404694    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:03.405176 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:03.404778    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:16:04.404247 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:04.404179    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:04.404379 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:04.404324    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:16:04.631477 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:04.631142    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:16:04.632558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:04.632121    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerStarted","Data":"283ccd3e539d7b3066572887b4ed4728404d35eb2490bf05c5b8bace83ef18d9"}
Sep 11 18:16:04.632558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:04.632533    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:16:04.632953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:04.632705    2578 scope.go:117] "RemoveContainer" containerID="343c0f4bab7d82513eaf2c909da67222ae9a41f3e2df92076f0480238e138e36"
Sep 11 18:16:04.650439 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:04.650316    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:16:05.404182 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.404145    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:05.404364 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:05.404250    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:16:05.404364 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.404306    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:05.404438 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:05.404384    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:16:05.637220 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.637195    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:16:05.637630 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.637531    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" event={"ID":"4e91f825-89cb-4019-b341-699c8c5a902d","Type":"ContainerStarted","Data":"5ce73e9f35a32bcb365046e54482a75091ed1fcd126c5ec8ff0e5d7f6d1186ee"}
Sep 11 18:16:05.637733 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.637716    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:16:05.637920 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.637907    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:16:05.639541 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.639519    2578 generic.go:356] "Generic (PLEG): container finished" podID="0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd" containerID="276dd2a8bbc32c8177fd10e271a935b3fa016ba099ed5be1cf7cdeedde588894" exitCode=0
Sep 11 18:16:05.639658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.639565    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-64z6w" event={"ID":"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd","Type":"ContainerDied","Data":"276dd2a8bbc32c8177fd10e271a935b3fa016ba099ed5be1cf7cdeedde588894"}
Sep 11 18:16:05.652986 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.652961    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:16:05.668270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:05.667224    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p" podStartSLOduration=8.87815523 podStartE2EDuration="27.66720714s" totalImagesPullingTime="18.78905191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:05.664492687 +0000 UTC m=+28.699709356" watchObservedRunningTime="2026-09-11 18:16:05.66720714 +0000 UTC m=+28.702423807"
Sep 11 18:16:06.403903 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:06.403837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:06.404045 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:06.403953    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:16:06.492004 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:06.491974    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bht8j"]
Sep 11 18:16:06.499007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:06.498982    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-czk7w"]
Sep 11 18:16:06.499159 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:06.499103    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:06.499253 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:06.499231    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:16:06.499592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:06.499522    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wkkbd"]
Sep 11 18:16:06.499705 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:06.499627    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:06.499784 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:06.499731    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:16:06.642395 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:06.642366    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:06.642975 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:06.642489    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:16:07.645990 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:07.645960    2578 generic.go:356] "Generic (PLEG): container finished" podID="0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd" containerID="2f592980a2724a989685327dc1f351ee9cb356df51b8d2db59ba862b1a4593c7" exitCode=0
Sep 11 18:16:07.646590 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:07.646366    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-64z6w" event={"ID":"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd","Type":"ContainerDied","Data":"2f592980a2724a989685327dc1f351ee9cb356df51b8d2db59ba862b1a4593c7"}
Sep 11 18:16:08.404488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:08.404458    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:08.404663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:08.404562    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:08.404663 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:08.404572    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:16:08.404663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:08.404598    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:08.404861 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:08.404731    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:16:08.404861 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:08.404830    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:16:08.651992 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:08.651908    2578 generic.go:356] "Generic (PLEG): container finished" podID="0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd" containerID="a2e1adbb733a9d6042f685ac5da28174412fd396d720ff29f61abe0767d00216" exitCode=0
Sep 11 18:16:08.651992 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:08.651979    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-64z6w" event={"ID":"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd","Type":"ContainerDied","Data":"a2e1adbb733a9d6042f685ac5da28174412fd396d720ff29f61abe0767d00216"}
Sep 11 18:16:10.404598 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:10.404398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:10.405062 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:10.404398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:10.405062 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:10.404712    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:16:10.405062 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:10.404398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:10.405062 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:10.404787    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wkkbd" podUID="6a6009bd-7d00-437f-9cf4-9161d79ab722"
Sep 11 18:16:10.405062 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:10.404852    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-czk7w" podUID="75588279-fbb8-4a19-a0c2-6ea50f680c22"
Sep 11 18:16:11.095316 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.095278    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:11.095545 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.095447    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:16:11.095545 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.095457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:11.095545 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.095517    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs podName:c3d9bcff-2714-44ac-8bd8-e288668a8a7b nodeName:}" failed. No retries permitted until 2026-09-11 18:16:43.095497232 +0000 UTC m=+66.130713898 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs") pod "network-metrics-daemon-bht8j" (UID: "c3d9bcff-2714-44ac-8bd8-e288668a8a7b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:16:11.095747 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.095594    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:16:11.095747 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.095613    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:16:11.095747 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.095628    2578 projected.go:196] Error preparing data for projected volume kube-api-access-q468p for pod openshift-network-diagnostics/network-check-target-czk7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:16:11.095747 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.095707    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p podName:75588279-fbb8-4a19-a0c2-6ea50f680c22 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:43.095692708 +0000 UTC m=+66.130909357 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q468p" (UniqueName: "kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p") pod "network-check-target-czk7w" (UID: "75588279-fbb8-4a19-a0c2-6ea50f680c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:16:11.195994 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.195958    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:11.196155 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.196132    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:16:11.196227 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.196216    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret podName:6a6009bd-7d00-437f-9cf4-9161d79ab722 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:43.196194617 +0000 UTC m=+66.231411282 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret") pod "global-pull-secret-syncer-wkkbd" (UID: "6a6009bd-7d00-437f-9cf4-9161d79ab722") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:16:11.299543 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.299457    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-82.ec2.internal" event="NodeReady"
Sep 11 18:16:11.299734 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.299591    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 18:16:11.369769 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.369724    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-88vkz"]
Sep 11 18:16:11.391531 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.391502    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5dh98"]
Sep 11 18:16:11.391705 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.391635    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.394757 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.394729    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 18:16:11.394903 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.394871    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ksdw9\""
Sep 11 18:16:11.394971 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.394906    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 18:16:11.407381 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.407362    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5dh98"]
Sep 11 18:16:11.407747 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.407392    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-88vkz"]
Sep 11 18:16:11.407747 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.407494    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5dh98"
Sep 11 18:16:11.413494 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.413473    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 18:16:11.413610 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.413535    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 18:16:11.413669 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.413619    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-f8nfl\""
Sep 11 18:16:11.413774 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.413758    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 18:16:11.497319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.497281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwcdm\" (UniqueName: \"kubernetes.io/projected/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-kube-api-access-fwcdm\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.497496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.497366    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmlj2\" (UniqueName: \"kubernetes.io/projected/2695bbaf-87ba-4022-8025-295c29f09e55-kube-api-access-vmlj2\") pod \"ingress-canary-5dh98\" (UID: \"2695bbaf-87ba-4022-8025-295c29f09e55\") " pod="openshift-ingress-canary/ingress-canary-5dh98"
Sep 11 18:16:11.497496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.497407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2695bbaf-87ba-4022-8025-295c29f09e55-cert\") pod \"ingress-canary-5dh98\" (UID: \"2695bbaf-87ba-4022-8025-295c29f09e55\") " pod="openshift-ingress-canary/ingress-canary-5dh98"
Sep 11 18:16:11.497496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.497435    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-config-volume\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.497630 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.497506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-tmp-dir\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.497630 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.497568    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.598614 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.598526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmlj2\" (UniqueName: \"kubernetes.io/projected/2695bbaf-87ba-4022-8025-295c29f09e55-kube-api-access-vmlj2\") pod \"ingress-canary-5dh98\" (UID: \"2695bbaf-87ba-4022-8025-295c29f09e55\") " pod="openshift-ingress-canary/ingress-canary-5dh98"
Sep 11 18:16:11.598801 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.598656    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2695bbaf-87ba-4022-8025-295c29f09e55-cert\") pod \"ingress-canary-5dh98\" (UID: \"2695bbaf-87ba-4022-8025-295c29f09e55\") " pod="openshift-ingress-canary/ingress-canary-5dh98"
Sep 11 18:16:11.598801 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.598707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-config-volume\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.598801 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.598726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-tmp-dir\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.598801 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.598772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.599039 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.598804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwcdm\" (UniqueName: \"kubernetes.io/projected/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-kube-api-access-fwcdm\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.599039 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.598920    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:16:11.599039 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:11.598984    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls podName:46bfe80b-cd69-4ba9-a7fb-128f20aa12d0 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:12.098966099 +0000 UTC m=+35.134182765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls") pod "dns-default-88vkz" (UID: "46bfe80b-cd69-4ba9-a7fb-128f20aa12d0") : secret "dns-default-metrics-tls" not found
Sep 11 18:16:11.599207 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.599095    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-tmp-dir\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.603447 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.603422    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2695bbaf-87ba-4022-8025-295c29f09e55-cert\") pod \"ingress-canary-5dh98\" (UID: \"2695bbaf-87ba-4022-8025-295c29f09e55\") " pod="openshift-ingress-canary/ingress-canary-5dh98"
Sep 11 18:16:11.609240 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.609202    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmlj2\" (UniqueName: \"kubernetes.io/projected/2695bbaf-87ba-4022-8025-295c29f09e55-kube-api-access-vmlj2\") pod \"ingress-canary-5dh98\" (UID: \"2695bbaf-87ba-4022-8025-295c29f09e55\") " pod="openshift-ingress-canary/ingress-canary-5dh98"
Sep 11 18:16:11.612665 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.612642    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-config-volume\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.615491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.615468    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwcdm\" (UniqueName: \"kubernetes.io/projected/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-kube-api-access-fwcdm\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:11.716689 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:11.716642    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5dh98"
Sep 11 18:16:12.102874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.102840    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:12.103053 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:12.102985    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:16:12.103053 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:12.103045    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls podName:46bfe80b-cd69-4ba9-a7fb-128f20aa12d0 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:13.103030143 +0000 UTC m=+36.138246790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls") pod "dns-default-88vkz" (UID: "46bfe80b-cd69-4ba9-a7fb-128f20aa12d0") : secret "dns-default-metrics-tls" not found
Sep 11 18:16:12.403991 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.403910    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:12.404146 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.403910    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:12.404199 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.403909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:12.408460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.408220    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 18:16:12.408460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.408235    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kq7d5\""
Sep 11 18:16:12.408460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.408275    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 18:16:12.408460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.408287    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 18:16:12.408460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.408235    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fhxbh\""
Sep 11 18:16:12.408966 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:12.408591    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 18:16:13.006477 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:13.005644    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5dh98"]
Sep 11 18:16:13.011273 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:16:13.011243    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2695bbaf_87ba_4022_8025_295c29f09e55.slice/crio-0461caec0aa436323a43f1071a35e6fcc764460883fe0700281c26b9d4171085 WatchSource:0}: Error finding container 0461caec0aa436323a43f1071a35e6fcc764460883fe0700281c26b9d4171085: Status 404 returned error can't find the container with id 0461caec0aa436323a43f1071a35e6fcc764460883fe0700281c26b9d4171085
Sep 11 18:16:13.110646 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:13.110494    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:13.110646 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:13.110637    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:16:13.110801 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:13.110717    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls podName:46bfe80b-cd69-4ba9-a7fb-128f20aa12d0 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:15.110702224 +0000 UTC m=+38.145918876 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls") pod "dns-default-88vkz" (UID: "46bfe80b-cd69-4ba9-a7fb-128f20aa12d0") : secret "dns-default-metrics-tls" not found
Sep 11 18:16:13.668613 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:13.668476    2578 generic.go:356] "Generic (PLEG): container finished" podID="0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd" containerID="4abb2deb5f1b147cdd1bea1a3207c23a0a098a15032bc9a942eeade101ba2217" exitCode=0
Sep 11 18:16:13.668613 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:13.668572    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-64z6w" event={"ID":"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd","Type":"ContainerDied","Data":"4abb2deb5f1b147cdd1bea1a3207c23a0a098a15032bc9a942eeade101ba2217"}
Sep 11 18:16:13.670104 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:13.670079    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5dh98" event={"ID":"2695bbaf-87ba-4022-8025-295c29f09e55","Type":"ContainerStarted","Data":"0461caec0aa436323a43f1071a35e6fcc764460883fe0700281c26b9d4171085"}
Sep 11 18:16:14.676720 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:14.676671    2578 generic.go:356] "Generic (PLEG): container finished" podID="0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd" containerID="7055d98821e9bfe65de288cf6ecdc78b70675f583b37846a30a3c7978f1b679d" exitCode=0
Sep 11 18:16:14.677181 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:14.676729    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-64z6w" event={"ID":"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd","Type":"ContainerDied","Data":"7055d98821e9bfe65de288cf6ecdc78b70675f583b37846a30a3c7978f1b679d"}
Sep 11 18:16:15.125910 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:15.125835    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:15.126041 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:15.125953    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:16:15.126041 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:15.126005    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls podName:46bfe80b-cd69-4ba9-a7fb-128f20aa12d0 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:19.12598971 +0000 UTC m=+42.161206358 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls") pod "dns-default-88vkz" (UID: "46bfe80b-cd69-4ba9-a7fb-128f20aa12d0") : secret "dns-default-metrics-tls" not found
Sep 11 18:16:15.680878 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:15.680838    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5dh98" event={"ID":"2695bbaf-87ba-4022-8025-295c29f09e55","Type":"ContainerStarted","Data":"48b97d1188c1079f2e7602e0e76a97fa03dba9c710c7b7483333e253b78b0031"}
Sep 11 18:16:15.684248 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:15.684220    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-64z6w" event={"ID":"0b1b8ab8-1bd7-4ebe-b274-9a9689afc6cd","Type":"ContainerStarted","Data":"3adf010f57a2ce5409a83abbe27887b1a90dbba5568043769ab15b719fde6f28"}
Sep 11 18:16:15.695725 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:15.695662    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5dh98" podStartSLOduration=2.3994171939999998 podStartE2EDuration="4.69564277s" totalImagesPullingTime="2.296225576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:15.695258155 +0000 UTC m=+38.730474828" watchObservedRunningTime="2026-09-11 18:16:15.69564277 +0000 UTC m=+38.730859438"
Sep 11 18:16:15.713520 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:15.713480    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-64z6w" podStartSLOduration=9.021036636 podStartE2EDuration="37.713465956s" totalImagesPullingTime="28.69242932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:15.712640713 +0000 UTC m=+38.747857385" watchObservedRunningTime="2026-09-11 18:16:15.713465956 +0000 UTC m=+38.748682663"
Sep 11 18:16:19.155457 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:19.155414    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:19.155996 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:19.155533    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:16:19.155996 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:19.155591    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls podName:46bfe80b-cd69-4ba9-a7fb-128f20aa12d0 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:27.155574774 +0000 UTC m=+50.190791439 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls") pod "dns-default-88vkz" (UID: "46bfe80b-cd69-4ba9-a7fb-128f20aa12d0") : secret "dns-default-metrics-tls" not found
Sep 11 18:16:27.207917 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:27.207872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:27.208337 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:27.207998    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:16:27.208337 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:27.208061    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls podName:46bfe80b-cd69-4ba9-a7fb-128f20aa12d0 nodeName:}" failed. No retries permitted until 2026-09-11 18:16:43.208045492 +0000 UTC m=+66.243262142 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls") pod "dns-default-88vkz" (UID: "46bfe80b-cd69-4ba9-a7fb-128f20aa12d0") : secret "dns-default-metrics-tls" not found
Sep 11 18:16:37.657454 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:37.657426    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hm89p"
Sep 11 18:16:43.116299 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.116252    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:16:43.116749 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.116329    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:43.119046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.119020    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 18:16:43.119170 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.119075    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 18:16:43.126718 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:43.126698    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 18:16:43.126788 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:43.126757    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs podName:c3d9bcff-2714-44ac-8bd8-e288668a8a7b nodeName:}" failed. No retries permitted until 2026-09-11 18:17:47.126737144 +0000 UTC m=+130.161953792 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs") pod "network-metrics-daemon-bht8j" (UID: "c3d9bcff-2714-44ac-8bd8-e288668a8a7b") : secret "metrics-daemon-secret" not found
Sep 11 18:16:43.128943 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.128926    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 18:16:43.140351 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.140328    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q468p\" (UniqueName: \"kubernetes.io/projected/75588279-fbb8-4a19-a0c2-6ea50f680c22-kube-api-access-q468p\") pod \"network-check-target-czk7w\" (UID: \"75588279-fbb8-4a19-a0c2-6ea50f680c22\") " pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:43.217504 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.217472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:43.217626 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.217556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:16:43.217663 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:43.217635    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:16:43.217726 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:16:43.217699    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls podName:46bfe80b-cd69-4ba9-a7fb-128f20aa12d0 nodeName:}" failed. No retries permitted until 2026-09-11 18:17:15.217671434 +0000 UTC m=+98.252888081 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls") pod "dns-default-88vkz" (UID: "46bfe80b-cd69-4ba9-a7fb-128f20aa12d0") : secret "dns-default-metrics-tls" not found
Sep 11 18:16:43.220184 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.220167    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 18:16:43.230591 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.230569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a6009bd-7d00-437f-9cf4-9161d79ab722-original-pull-secret\") pod \"global-pull-secret-syncer-wkkbd\" (UID: \"6a6009bd-7d00-437f-9cf4-9161d79ab722\") " pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:43.316072 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.316044    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wkkbd"
Sep 11 18:16:43.324873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.324849    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fhxbh\""
Sep 11 18:16:43.332808 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.332787    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:43.611805 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.611773    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wkkbd"]
Sep 11 18:16:43.616214 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:16:43.616184    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a6009bd_7d00_437f_9cf4_9161d79ab722.slice/crio-6692d357c13824a582ef2f1e2f8f46374cfa632901556afbde6bb99964d5e3c5 WatchSource:0}: Error finding container 6692d357c13824a582ef2f1e2f8f46374cfa632901556afbde6bb99964d5e3c5: Status 404 returned error can't find the container with id 6692d357c13824a582ef2f1e2f8f46374cfa632901556afbde6bb99964d5e3c5
Sep 11 18:16:43.642326 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.642303    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-czk7w"]
Sep 11 18:16:43.645260 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:16:43.645236    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75588279_fbb8_4a19_a0c2_6ea50f680c22.slice/crio-b00e3e3c83a25ee296614ea909b63ae7abce83556e34f98773c38672bcc7caeb WatchSource:0}: Error finding container b00e3e3c83a25ee296614ea909b63ae7abce83556e34f98773c38672bcc7caeb: Status 404 returned error can't find the container with id b00e3e3c83a25ee296614ea909b63ae7abce83556e34f98773c38672bcc7caeb
Sep 11 18:16:43.753934 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.753901    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-czk7w" event={"ID":"75588279-fbb8-4a19-a0c2-6ea50f680c22","Type":"ContainerStarted","Data":"b00e3e3c83a25ee296614ea909b63ae7abce83556e34f98773c38672bcc7caeb"}
Sep 11 18:16:43.755026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:43.755002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wkkbd" event={"ID":"6a6009bd-7d00-437f-9cf4-9161d79ab722","Type":"ContainerStarted","Data":"6692d357c13824a582ef2f1e2f8f46374cfa632901556afbde6bb99964d5e3c5"}
Sep 11 18:16:48.770111 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:48.770076    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-czk7w" event={"ID":"75588279-fbb8-4a19-a0c2-6ea50f680c22","Type":"ContainerStarted","Data":"1920c102d85bbaaecff4c98a32c93c15c6f24cfbd59fff30e1f61f29a51e526e"}
Sep 11 18:16:48.770607 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:48.770203    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:16:48.771625 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:48.771602    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wkkbd" event={"ID":"6a6009bd-7d00-437f-9cf4-9161d79ab722","Type":"ContainerStarted","Data":"6283259ba482a93c2839108d9437195d8c1e00dacab42cadf166200abd66f38f"}
Sep 11 18:16:48.786662 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:48.786617    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-czk7w" podStartSLOduration=66.465307608 podStartE2EDuration="1m10.786603847s" totalImagesPullingTime="4.321296239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:48.786149481 +0000 UTC m=+71.821366152" watchObservedRunningTime="2026-09-11 18:16:48.786603847 +0000 UTC m=+71.821820508"
Sep 11 18:16:48.800323 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:16:48.800260    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wkkbd" podStartSLOduration=66.446777528 podStartE2EDuration="1m10.800242544s" totalImagesPullingTime="4.353465016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:48.799483702 +0000 UTC m=+71.834700371" watchObservedRunningTime="2026-09-11 18:16:48.800242544 +0000 UTC m=+71.835459215"
Sep 11 18:17:15.225945 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:15.225906    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:17:15.226339 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:15.226027    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:17:15.226339 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:15.226086    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls podName:46bfe80b-cd69-4ba9-a7fb-128f20aa12d0 nodeName:}" failed. No retries permitted until 2026-09-11 18:18:19.226071156 +0000 UTC m=+162.261287810 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls") pod "dns-default-88vkz" (UID: "46bfe80b-cd69-4ba9-a7fb-128f20aa12d0") : secret "dns-default-metrics-tls" not found
Sep 11 18:17:19.776377 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:19.776338    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-czk7w"
Sep 11 18:17:44.460919 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.460881    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"]
Sep 11 18:17:44.462795 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.462779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.465226 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.465207    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:17:44.465340 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.465205    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 18:17:44.466098 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.466081    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 18:17:44.466153 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.466126    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-25brg\""
Sep 11 18:17:44.466201 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.466191    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 18:17:44.470116 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.470096    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7bf46bd65d-jtnbl"]
Sep 11 18:17:44.472192 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.472170    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-gxksc"]
Sep 11 18:17:44.472305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.472275    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.473861 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.473839    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-dbst7"]
Sep 11 18:17:44.473964 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.473935    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.474439 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.474422    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 11 18:17:44.475939 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.475919    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"]
Sep 11 18:17:44.476056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.476010    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.479919 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.479901    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 11 18:17:44.480125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480104    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 18:17:44.480233 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480213    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 18:17:44.480390 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480369    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 11 18:17:44.480494 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480441    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 18:17:44.480494 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480452    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 18:17:44.480494 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480452    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jf6r4\""
Sep 11 18:17:44.480646 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480441    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:17:44.480646 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480467    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 18:17:44.480646 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480502    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 18:17:44.480775 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480667    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 18:17:44.480775 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480699    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 11 18:17:44.480775 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480765    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 18:17:44.480868 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.480768    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 18:17:44.481113 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.481097    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2ffgj\""
Sep 11 18:17:44.481257 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.481242    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gk6bn\""
Sep 11 18:17:44.486444 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.486425    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 18:17:44.486941 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.486925    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 18:17:44.493563 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.493543    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-gxksc"]
Sep 11 18:17:44.494220 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.494199    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7bf46bd65d-jtnbl"]
Sep 11 18:17:44.504116 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.504091    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-dbst7"]
Sep 11 18:17:44.617145 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvs65\" (UniqueName: \"kubernetes.io/projected/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-kube-api-access-xvs65\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.617145 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c1b338a0-0878-41c0-9bf6-13d543b94c2e-config\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.617360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617159    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c1b338a0-0878-41c0-9bf6-13d543b94c2e-serving-cert\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.617360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617184    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4bnq\" (UniqueName: \"kubernetes.io/projected/f1d12737-e1eb-4321-9473-e06391ac2642-kube-api-access-b4bnq\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.617360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617208    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-tmp\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.617360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617232    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-default-certificate\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.617360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-stats-auth\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.617360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1b338a0-0878-41c0-9bf6-13d543b94c2e-trusted-ca\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.617360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617344    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7-serving-cert\") pod \"service-ca-operator-cf7c776b7-cr7cw\" (UID: \"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.617585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617376    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-service-ca-bundle\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.617585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617399    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-trusted-ca-bundle\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.617585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.617585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617448    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dcz9\" (UniqueName: \"kubernetes.io/projected/c1b338a0-0878-41c0-9bf6-13d543b94c2e-kube-api-access-2dcz9\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.617585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617479    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7-config\") pod \"service-ca-operator-cf7c776b7-cr7cw\" (UID: \"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.617585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-snapshots\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.617585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617519    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfmht\" (UniqueName: \"kubernetes.io/projected/818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7-kube-api-access-sfmht\") pod \"service-ca-operator-cf7c776b7-cr7cw\" (UID: \"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.617585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-serving-cert\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.617585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.617557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.717910 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.717820    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-tmp\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.717910 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.717869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-default-certificate\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.717910 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.717888    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-stats-auth\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.717910 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.717903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1b338a0-0878-41c0-9bf6-13d543b94c2e-trusted-ca\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.717910 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.717920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7-serving-cert\") pod \"service-ca-operator-cf7c776b7-cr7cw\" (UID: \"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.718285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718075    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-service-ca-bundle\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.718285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-trusted-ca-bundle\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.718285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.718285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dcz9\" (UniqueName: \"kubernetes.io/projected/c1b338a0-0878-41c0-9bf6-13d543b94c2e-kube-api-access-2dcz9\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.718285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7-config\") pod \"service-ca-operator-cf7c776b7-cr7cw\" (UID: \"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.718285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-snapshots\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.718285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfmht\" (UniqueName: \"kubernetes.io/projected/818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7-kube-api-access-sfmht\") pod \"service-ca-operator-cf7c776b7-cr7cw\" (UID: \"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.718584 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-serving-cert\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.718584 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.718584 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718334    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-tmp\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.718584 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvs65\" (UniqueName: \"kubernetes.io/projected/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-kube-api-access-xvs65\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.719119 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.719094    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7-config\") pod \"service-ca-operator-cf7c776b7-cr7cw\" (UID: \"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.719224 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.719207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1b338a0-0878-41c0-9bf6-13d543b94c2e-trusted-ca\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.719612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.719587    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-service-ca-bundle\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.719669 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.718420    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c1b338a0-0878-41c0-9bf6-13d543b94c2e-config\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.719748 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.719700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c1b338a0-0878-41c0-9bf6-13d543b94c2e-serving-cert\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.719794 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.719755    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4bnq\" (UniqueName: \"kubernetes.io/projected/f1d12737-e1eb-4321-9473-e06391ac2642-kube-api-access-b4bnq\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.721203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-stats-auth\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.721246    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-default-certificate\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.721557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7-serving-cert\") pod \"service-ca-operator-cf7c776b7-cr7cw\" (UID: \"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.722169    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c1b338a0-0878-41c0-9bf6-13d543b94c2e-config\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:44.722360    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:17:45.222298018 +0000 UTC m=+128.257514686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : configmap references non-existent config key: service-ca.crt
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:44.722639    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:44.722729    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:17:45.222711643 +0000 UTC m=+128.257928291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : secret "router-metrics-certs-default" not found
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.722948    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-snapshots\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.722993    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c1b338a0-0878-41c0-9bf6-13d543b94c2e-serving-cert\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.723818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.723051    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-trusted-ca-bundle\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.724332 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.724147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-serving-cert\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.727419 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.727371    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4bnq\" (UniqueName: \"kubernetes.io/projected/f1d12737-e1eb-4321-9473-e06391ac2642-kube-api-access-b4bnq\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:44.728899 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.728876    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfmht\" (UniqueName: \"kubernetes.io/projected/818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7-kube-api-access-sfmht\") pod \"service-ca-operator-cf7c776b7-cr7cw\" (UID: \"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.729203 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.729186    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dcz9\" (UniqueName: \"kubernetes.io/projected/c1b338a0-0878-41c0-9bf6-13d543b94c2e-kube-api-access-2dcz9\") pod \"console-operator-696884fb9c-gxksc\" (UID: \"c1b338a0-0878-41c0-9bf6-13d543b94c2e\") " pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.732267 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.732242    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvs65\" (UniqueName: \"kubernetes.io/projected/566a7849-1b81-4d2d-b3ad-80cc5d8031dc-kube-api-access-xvs65\") pod \"insights-operator-5844b856db-dbst7\" (UID: \"566a7849-1b81-4d2d-b3ad-80cc5d8031dc\") " pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:44.771275 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.771248    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"
Sep 11 18:17:44.788123 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.788103    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:44.793248 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:44.793227    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-dbst7"
Sep 11 18:17:45.047894 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:45.047817    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw"]
Sep 11 18:17:45.050829 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:17:45.050802    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818f3ffd_a8f8_4a3d_b234_b330a5fd8cc7.slice/crio-0d162fa13b915d31073fd29adb50a330c62014d44f0f8d41798e87b43eeaeefa WatchSource:0}: Error finding container 0d162fa13b915d31073fd29adb50a330c62014d44f0f8d41798e87b43eeaeefa: Status 404 returned error can't find the container with id 0d162fa13b915d31073fd29adb50a330c62014d44f0f8d41798e87b43eeaeefa
Sep 11 18:17:45.070103 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:45.070058    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-gxksc"]
Sep 11 18:17:45.074122 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:17:45.074092    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b338a0_0878_41c0_9bf6_13d543b94c2e.slice/crio-e53d3b3d10d25634c770ad84a7bd764579421b8d5b2dad2c1fc0d7608501c5dd WatchSource:0}: Error finding container e53d3b3d10d25634c770ad84a7bd764579421b8d5b2dad2c1fc0d7608501c5dd: Status 404 returned error can't find the container with id e53d3b3d10d25634c770ad84a7bd764579421b8d5b2dad2c1fc0d7608501c5dd
Sep 11 18:17:45.074226 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:45.074160    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-dbst7"]
Sep 11 18:17:45.077977 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:17:45.077955    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod566a7849_1b81_4d2d_b3ad_80cc5d8031dc.slice/crio-28a11749a338e7deeafc22b2a8c24953a9dbf8c320827dbb5e71e12fc8a95cf6 WatchSource:0}: Error finding container 28a11749a338e7deeafc22b2a8c24953a9dbf8c320827dbb5e71e12fc8a95cf6: Status 404 returned error can't find the container with id 28a11749a338e7deeafc22b2a8c24953a9dbf8c320827dbb5e71e12fc8a95cf6
Sep 11 18:17:45.224436 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:45.224403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:45.224603 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:45.224447    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:45.224603 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:45.224551    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 18:17:45.224603 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:45.224567    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:17:46.224549095 +0000 UTC m=+129.259765743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : configmap references non-existent config key: service-ca.crt
Sep 11 18:17:45.224603 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:45.224593    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:17:46.224582635 +0000 UTC m=+129.259799283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : secret "router-metrics-certs-default" not found
Sep 11 18:17:45.910696 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:45.910644    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-dbst7" event={"ID":"566a7849-1b81-4d2d-b3ad-80cc5d8031dc","Type":"ContainerStarted","Data":"28a11749a338e7deeafc22b2a8c24953a9dbf8c320827dbb5e71e12fc8a95cf6"}
Sep 11 18:17:45.912222 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:45.912177    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw" event={"ID":"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7","Type":"ContainerStarted","Data":"0d162fa13b915d31073fd29adb50a330c62014d44f0f8d41798e87b43eeaeefa"}
Sep 11 18:17:45.913708 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:45.913657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gxksc" event={"ID":"c1b338a0-0878-41c0-9bf6-13d543b94c2e","Type":"ContainerStarted","Data":"e53d3b3d10d25634c770ad84a7bd764579421b8d5b2dad2c1fc0d7608501c5dd"}
Sep 11 18:17:46.234018 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:46.233933    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:46.234018 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:46.233980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:46.234239 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:46.234096    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 18:17:46.234239 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:46.234126    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:17:48.234102589 +0000 UTC m=+131.269319238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : configmap references non-existent config key: service-ca.crt
Sep 11 18:17:46.234239 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:46.234154    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:17:48.234145814 +0000 UTC m=+131.269362465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : secret "router-metrics-certs-default" not found
Sep 11 18:17:47.141363 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:47.141316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:17:47.141777 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:47.141506    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 18:17:47.141777 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:47.141588    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs podName:c3d9bcff-2714-44ac-8bd8-e288668a8a7b nodeName:}" failed. No retries permitted until 2026-09-11 18:19:49.141565136 +0000 UTC m=+252.176781788 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs") pod "network-metrics-daemon-bht8j" (UID: "c3d9bcff-2714-44ac-8bd8-e288668a8a7b") : secret "metrics-daemon-secret" not found
Sep 11 18:17:48.249056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.248977    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:48.249056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.249027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:48.249455 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:48.249139    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:17:52.24912294 +0000 UTC m=+135.284339605 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : configmap references non-existent config key: service-ca.crt
Sep 11 18:17:48.249455 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:48.249219    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 18:17:48.249455 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:48.249289    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:17:52.249271757 +0000 UTC m=+135.284488421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : secret "router-metrics-certs-default" not found
Sep 11 18:17:48.470467 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.470430    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z"]
Sep 11 18:17:48.472299 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.472283    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z"
Sep 11 18:17:48.474636 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.474613    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 18:17:48.474888 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.474870    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 18:17:48.475480 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.475464    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vz28j\""
Sep 11 18:17:48.487185 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.487159    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z"]
Sep 11 18:17:48.551174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.551067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnbcx\" (UniqueName: \"kubernetes.io/projected/e6dc24b4-e75b-4a74-a189-7ffb153be3ef-kube-api-access-dnbcx\") pod \"migrator-8b4d697c5-psl4z\" (UID: \"e6dc24b4-e75b-4a74-a189-7ffb153be3ef\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z"
Sep 11 18:17:48.651828 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.651781    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnbcx\" (UniqueName: \"kubernetes.io/projected/e6dc24b4-e75b-4a74-a189-7ffb153be3ef-kube-api-access-dnbcx\") pod \"migrator-8b4d697c5-psl4z\" (UID: \"e6dc24b4-e75b-4a74-a189-7ffb153be3ef\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z"
Sep 11 18:17:48.661031 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.661001    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnbcx\" (UniqueName: \"kubernetes.io/projected/e6dc24b4-e75b-4a74-a189-7ffb153be3ef-kube-api-access-dnbcx\") pod \"migrator-8b4d697c5-psl4z\" (UID: \"e6dc24b4-e75b-4a74-a189-7ffb153be3ef\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z"
Sep 11 18:17:48.782590 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.782550    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z"
Sep 11 18:17:48.924650 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.924608    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-dbst7" event={"ID":"566a7849-1b81-4d2d-b3ad-80cc5d8031dc","Type":"ContainerStarted","Data":"58914f934a31d02b98678349b248f3b065012b07883014e3c8bd1fbe16ee7b86"}
Sep 11 18:17:48.926231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.926199    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw" event={"ID":"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7","Type":"ContainerStarted","Data":"c4c1c6c257f480dd43f9b5a0f315017a52d280b9c426bdfce59652891ac789fb"}
Sep 11 18:17:48.927978 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.927954    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/0.log"
Sep 11 18:17:48.928121 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.927995    2578 generic.go:356] "Generic (PLEG): container finished" podID="c1b338a0-0878-41c0-9bf6-13d543b94c2e" containerID="9085b3d51018fb5cc85caa640dd18c18b2305210ebc98aab1587eea42128c57b" exitCode=255
Sep 11 18:17:48.928121 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.928028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gxksc" event={"ID":"c1b338a0-0878-41c0-9bf6-13d543b94c2e","Type":"ContainerDied","Data":"9085b3d51018fb5cc85caa640dd18c18b2305210ebc98aab1587eea42128c57b"}
Sep 11 18:17:48.928324 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.928307    2578 scope.go:117] "RemoveContainer" containerID="9085b3d51018fb5cc85caa640dd18c18b2305210ebc98aab1587eea42128c57b"
Sep 11 18:17:48.947063 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.947013    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-dbst7" podStartSLOduration=2.032830321 podStartE2EDuration="4.946992433s" totalImagesPullingTime="2.914162112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:17:48.946063066 +0000 UTC m=+131.981279737" watchObservedRunningTime="2026-09-11 18:17:48.946992433 +0000 UTC m=+131.982209104"
Sep 11 18:17:48.961346 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:48.961296    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw" podStartSLOduration=2.02451574 podStartE2EDuration="4.961258115s" totalImagesPullingTime="2.936742375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:17:48.960557078 +0000 UTC m=+131.995773770" watchObservedRunningTime="2026-09-11 18:17:48.961258115 +0000 UTC m=+131.996474785"
Sep 11 18:17:49.063367 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.063340    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z"]
Sep 11 18:17:49.113570 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:17:49.113499    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6dc24b4_e75b_4a74_a189_7ffb153be3ef.slice/crio-8c9e18cc5a17967a2a183cb31b3ad3ab1d5ed62ff3c692ace20b7ddc6c0bfa07 WatchSource:0}: Error finding container 8c9e18cc5a17967a2a183cb31b3ad3ab1d5ed62ff3c692ace20b7ddc6c0bfa07: Status 404 returned error can't find the container with id 8c9e18cc5a17967a2a183cb31b3ad3ab1d5ed62ff3c692ace20b7ddc6c0bfa07
Sep 11 18:17:49.302281 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.302242    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hmtmq"]
Sep 11 18:17:49.304651 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.304626    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.306997 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.306978    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 18:17:49.307100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.307010    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-psjlx\""
Sep 11 18:17:49.307100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.307066    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 18:17:49.315949 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.315922    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hmtmq"]
Sep 11 18:17:49.457437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.457407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-crio-socket\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.457616 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.457445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l9gn\" (UniqueName: \"kubernetes.io/projected/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-api-access-7l9gn\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.457616 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.457498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-data-volume\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.457616 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.457530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.457744 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.457623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.558529 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.558491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-data-volume\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.558529 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.558537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.558786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.558574    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.558786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.558631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-crio-socket\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.558786 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:49.558638    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:17:49.558786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.558728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-crio-socket\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.558786 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:49.558738    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:17:50.0587142 +0000 UTC m=+133.093930856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : configmap "kube-rbac-proxy" not found
Sep 11 18:17:49.558786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.558649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9gn\" (UniqueName: \"kubernetes.io/projected/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-api-access-7l9gn\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.558786 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:49.558767    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:17:49.559069 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:49.558825    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:17:50.058810493 +0000 UTC m=+133.094027140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:17:49.559069 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.558928    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-data-volume\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.567422 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.567404    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l9gn\" (UniqueName: \"kubernetes.io/projected/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-api-access-7l9gn\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:49.933560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.933530    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:17:49.934149 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.934121    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/0.log"
Sep 11 18:17:49.934293 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.934168    2578 generic.go:356] "Generic (PLEG): container finished" podID="c1b338a0-0878-41c0-9bf6-13d543b94c2e" containerID="8d3b2981d39b169314d52ade3f318397b6ca83938bb7119338ba4dd0b6de3460" exitCode=255
Sep 11 18:17:49.934293 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.934259    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gxksc" event={"ID":"c1b338a0-0878-41c0-9bf6-13d543b94c2e","Type":"ContainerDied","Data":"8d3b2981d39b169314d52ade3f318397b6ca83938bb7119338ba4dd0b6de3460"}
Sep 11 18:17:49.934841 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.934821    2578 scope.go:117] "RemoveContainer" containerID="9085b3d51018fb5cc85caa640dd18c18b2305210ebc98aab1587eea42128c57b"
Sep 11 18:17:49.934968 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.934934    2578 scope.go:117] "RemoveContainer" containerID="8d3b2981d39b169314d52ade3f318397b6ca83938bb7119338ba4dd0b6de3460"
Sep 11 18:17:49.935194 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:49.935168    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-gxksc_openshift-console-operator(c1b338a0-0878-41c0-9bf6-13d543b94c2e)\"" pod="openshift-console-operator/console-operator-696884fb9c-gxksc" podUID="c1b338a0-0878-41c0-9bf6-13d543b94c2e"
Sep 11 18:17:49.936020 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:49.935954    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z" event={"ID":"e6dc24b4-e75b-4a74-a189-7ffb153be3ef","Type":"ContainerStarted","Data":"8c9e18cc5a17967a2a183cb31b3ad3ab1d5ed62ff3c692ace20b7ddc6c0bfa07"}
Sep 11 18:17:50.061756 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:50.061720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:50.061941 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:50.061880    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:17:50.061999 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:50.061956    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:17:51.061934944 +0000 UTC m=+134.097151617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:17:50.062239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:50.062092    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:50.062239 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:50.062176    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:17:50.062239 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:50.062216    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:17:51.06220413 +0000 UTC m=+134.097420804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : configmap "kube-rbac-proxy" not found
Sep 11 18:17:50.940546 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:50.940517    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:17:50.941059 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:50.940914    2578 scope.go:117] "RemoveContainer" containerID="8d3b2981d39b169314d52ade3f318397b6ca83938bb7119338ba4dd0b6de3460"
Sep 11 18:17:50.941115 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:50.941089    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-gxksc_openshift-console-operator(c1b338a0-0878-41c0-9bf6-13d543b94c2e)\"" pod="openshift-console-operator/console-operator-696884fb9c-gxksc" podUID="c1b338a0-0878-41c0-9bf6-13d543b94c2e"
Sep 11 18:17:50.942434 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:50.942411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z" event={"ID":"e6dc24b4-e75b-4a74-a189-7ffb153be3ef","Type":"ContainerStarted","Data":"f63b9a9f011bb619c09b8b2c777b04e95ea150d10ec6573497b3b3787511f0a7"}
Sep 11 18:17:50.942561 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:50.942477    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z" event={"ID":"e6dc24b4-e75b-4a74-a189-7ffb153be3ef","Type":"ContainerStarted","Data":"0b0edda528ffc5357967f8d0121101c7039cafa46c95d9ab13a9e8035aa63a83"}
Sep 11 18:17:50.970381 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:50.970333    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-psl4z" podStartSLOduration=1.586300789 podStartE2EDuration="2.97031642s" totalImagesPullingTime="1.384015631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:17:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:17:50.970143267 +0000 UTC m=+134.005359936" watchObservedRunningTime="2026-09-11 18:17:50.97031642 +0000 UTC m=+134.005533092"
Sep 11 18:17:51.071287 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:51.071244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:51.071501 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:51.071312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:51.071501 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:51.071429    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:17:51.071501 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:51.071469    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:17:51.071716 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:51.071505    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:17:53.071484029 +0000 UTC m=+136.106700693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : configmap "kube-rbac-proxy" not found
Sep 11 18:17:51.071716 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:51.071532    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:17:53.071513248 +0000 UTC m=+136.106729899 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:17:52.253414 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:52.253384    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v9sp7_2a2454c2-8f32-4025-9700-5c3464f05e73/dns-node-resolver/0.log"
Sep 11 18:17:52.282054 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:52.282020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:52.282210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:52.282120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:17:52.282210 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:52.282175    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 18:17:52.282279 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:52.282238    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:18:00.282224306 +0000 UTC m=+143.317440961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : secret "router-metrics-certs-default" not found
Sep 11 18:17:52.282279 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:52.282253    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:18:00.282247584 +0000 UTC m=+143.317464232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : configmap references non-existent config key: service-ca.crt
Sep 11 18:17:52.852059 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:52.852024    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d8828_0d5ec55f-0d86-44fc-adb6-b562c0d6f298/node-ca/0.log"
Sep 11 18:17:53.088650 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:53.088613    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:53.088833 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:53.088694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:53.088833 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:53.088757    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:17:53.088833 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:53.088826    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:17:57.088809316 +0000 UTC m=+140.124025970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : configmap "kube-rbac-proxy" not found
Sep 11 18:17:53.088950 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:53.088841    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:17:53.088950 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:53.088878    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:17:57.088866493 +0000 UTC m=+140.124083155 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:17:53.851591 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:53.851549    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5dh98_2695bbaf-87ba-4022-8025-295c29f09e55/serve-healthcheck-canary/0.log"
Sep 11 18:17:54.252912 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:54.252882    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-psl4z_e6dc24b4-e75b-4a74-a189-7ffb153be3ef/migrator/0.log"
Sep 11 18:17:54.453503 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:54.453470    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-psl4z_e6dc24b4-e75b-4a74-a189-7ffb153be3ef/graceful-termination/0.log"
Sep 11 18:17:54.788880 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:54.788844    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:54.789061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:54.788915    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:17:54.789265 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:54.789253    2578 scope.go:117] "RemoveContainer" containerID="8d3b2981d39b169314d52ade3f318397b6ca83938bb7119338ba4dd0b6de3460"
Sep 11 18:17:54.789424 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:54.789407    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-gxksc_openshift-console-operator(c1b338a0-0878-41c0-9bf6-13d543b94c2e)\"" pod="openshift-console-operator/console-operator-696884fb9c-gxksc" podUID="c1b338a0-0878-41c0-9bf6-13d543b94c2e"
Sep 11 18:17:57.124260 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:57.124216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:57.124701 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:17:57.124282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:17:57.124701 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:57.124334    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:17:57.124701 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:57.124396    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:17:57.124701 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:57.124404    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:18:05.124387127 +0000 UTC m=+148.159603776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : configmap "kube-rbac-proxy" not found
Sep 11 18:17:57.124701 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:17:57.124436    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:18:05.124425517 +0000 UTC m=+148.159642166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:18:00.351282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:00.351225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:00.351839 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:00.351352    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:00.351839 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:00.351492    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle podName:f1d12737-e1eb-4321-9473-e06391ac2642 nodeName:}" failed. No retries permitted until 2026-09-11 18:18:16.351474122 +0000 UTC m=+159.386690769 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle") pod "router-default-7bf46bd65d-jtnbl" (UID: "f1d12737-e1eb-4321-9473-e06391ac2642") : configmap references non-existent config key: service-ca.crt
Sep 11 18:18:00.353583 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:00.353561    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1d12737-e1eb-4321-9473-e06391ac2642-metrics-certs\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:05.188880 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:05.188840    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:18:05.189308 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:05.188897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:18:05.189308 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:05.189024    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:18:05.189308 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:05.189077    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls podName:9bbc029c-d8f7-46bc-bfe9-a55922054b8b nodeName:}" failed. No retries permitted until 2026-09-11 18:18:21.189063922 +0000 UTC m=+164.224280571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmtmq" (UID: "9bbc029c-d8f7-46bc-bfe9-a55922054b8b") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:18:05.189426 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:05.189373    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:18:08.404757 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:08.404725    2578 scope.go:117] "RemoveContainer" containerID="8d3b2981d39b169314d52ade3f318397b6ca83938bb7119338ba4dd0b6de3460"
Sep 11 18:18:09.000953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:09.000924    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:18:09.001120 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:09.001000    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-gxksc" event={"ID":"c1b338a0-0878-41c0-9bf6-13d543b94c2e","Type":"ContainerStarted","Data":"fcbd1d5736e5c261e5571697c1a20002c62dc50d6768996f409059d5febea570"}
Sep 11 18:18:09.001326 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:09.001294    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:18:09.008712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:09.008658    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-gxksc"
Sep 11 18:18:09.020719 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:09.020643    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-gxksc" podStartSLOduration=22.100935473 podStartE2EDuration="25.020627677s" totalImagesPullingTime="2.919692204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:09.019842868 +0000 UTC m=+152.055059537" watchObservedRunningTime="2026-09-11 18:18:09.020627677 +0000 UTC m=+152.055844348"
Sep 11 18:18:10.650923 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.650893    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"]
Sep 11 18:18:10.653576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.653558    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"
Sep 11 18:18:10.657277 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.657240    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 18:18:10.657277 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.657241    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 18:18:10.657539 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.657521    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 11 18:18:10.658112 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.658093    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 18:18:10.658857 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.658841    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-554d9\""
Sep 11 18:18:10.678777 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.678744    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"]
Sep 11 18:18:10.727394 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.727360    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73faf9e3-65f8-4816-bdee-82b1e8a0227a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dc495bb6-zrwbm\" (UID: \"73faf9e3-65f8-4816-bdee-82b1e8a0227a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"
Sep 11 18:18:10.727585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.727416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhlgt\" (UniqueName: \"kubernetes.io/projected/73faf9e3-65f8-4816-bdee-82b1e8a0227a-kube-api-access-rhlgt\") pod \"managed-serviceaccount-addon-agent-6dc495bb6-zrwbm\" (UID: \"73faf9e3-65f8-4816-bdee-82b1e8a0227a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"
Sep 11 18:18:10.729815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.729787    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68756467d4-gmxt5"]
Sep 11 18:18:10.732788 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.732766    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.741121 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.741101    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 18:18:10.741487 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.741471    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 18:18:10.742561 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.742545    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 18:18:10.745820 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.745805    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-grmtt\""
Sep 11 18:18:10.761854 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.761818    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 18:18:10.790162 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.790127    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68756467d4-gmxt5"]
Sep 11 18:18:10.828244 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828211    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a936883-9636-4760-9260-27a0c9a7731b-image-registry-private-configuration\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.828244 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828244    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6hgn\" (UniqueName: \"kubernetes.io/projected/5a936883-9636-4760-9260-27a0c9a7731b-kube-api-access-m6hgn\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.828503 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a936883-9636-4760-9260-27a0c9a7731b-bound-sa-token\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.828503 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a936883-9636-4760-9260-27a0c9a7731b-installation-pull-secrets\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.828503 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828342    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a936883-9636-4760-9260-27a0c9a7731b-ca-trust-extracted\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.828503 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a936883-9636-4760-9260-27a0c9a7731b-registry-tls\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.828503 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73faf9e3-65f8-4816-bdee-82b1e8a0227a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dc495bb6-zrwbm\" (UID: \"73faf9e3-65f8-4816-bdee-82b1e8a0227a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"
Sep 11 18:18:10.828810 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828513    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a936883-9636-4760-9260-27a0c9a7731b-trusted-ca\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.828810 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a936883-9636-4760-9260-27a0c9a7731b-registry-certificates\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.828810 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.828582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhlgt\" (UniqueName: \"kubernetes.io/projected/73faf9e3-65f8-4816-bdee-82b1e8a0227a-kube-api-access-rhlgt\") pod \"managed-serviceaccount-addon-agent-6dc495bb6-zrwbm\" (UID: \"73faf9e3-65f8-4816-bdee-82b1e8a0227a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"
Sep 11 18:18:10.831216 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.831191    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73faf9e3-65f8-4816-bdee-82b1e8a0227a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dc495bb6-zrwbm\" (UID: \"73faf9e3-65f8-4816-bdee-82b1e8a0227a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"
Sep 11 18:18:10.851064 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.851025    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhlgt\" (UniqueName: \"kubernetes.io/projected/73faf9e3-65f8-4816-bdee-82b1e8a0227a-kube-api-access-rhlgt\") pod \"managed-serviceaccount-addon-agent-6dc495bb6-zrwbm\" (UID: \"73faf9e3-65f8-4816-bdee-82b1e8a0227a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"
Sep 11 18:18:10.929711 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.929653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a936883-9636-4760-9260-27a0c9a7731b-installation-pull-secrets\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.929711 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.929712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a936883-9636-4760-9260-27a0c9a7731b-ca-trust-extracted\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.929972 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.929736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a936883-9636-4760-9260-27a0c9a7731b-registry-tls\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.929972 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.929768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a936883-9636-4760-9260-27a0c9a7731b-trusted-ca\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.929972 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.929795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a936883-9636-4760-9260-27a0c9a7731b-registry-certificates\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.930335 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.930242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a936883-9636-4760-9260-27a0c9a7731b-image-registry-private-configuration\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.930760 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.930332    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6hgn\" (UniqueName: \"kubernetes.io/projected/5a936883-9636-4760-9260-27a0c9a7731b-kube-api-access-m6hgn\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.931231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.931071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a936883-9636-4760-9260-27a0c9a7731b-bound-sa-token\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.931231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.931109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a936883-9636-4760-9260-27a0c9a7731b-ca-trust-extracted\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.931231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.931154    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a936883-9636-4760-9260-27a0c9a7731b-registry-certificates\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.933263 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.933223    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a936883-9636-4760-9260-27a0c9a7731b-trusted-ca\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.935818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.933449    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a936883-9636-4760-9260-27a0c9a7731b-installation-pull-secrets\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.935818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.933592    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a936883-9636-4760-9260-27a0c9a7731b-image-registry-private-configuration\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.936738 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.936713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a936883-9636-4760-9260-27a0c9a7731b-registry-tls\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.942361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.942334    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a936883-9636-4760-9260-27a0c9a7731b-bound-sa-token\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.942868 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.942852    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6hgn\" (UniqueName: \"kubernetes.io/projected/5a936883-9636-4760-9260-27a0c9a7731b-kube-api-access-m6hgn\") pod \"image-registry-68756467d4-gmxt5\" (UID: \"5a936883-9636-4760-9260-27a0c9a7731b\") " pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:10.979999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:10.979960    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"
Sep 11 18:18:11.041102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:11.041067    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:11.252510 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:11.252422    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm"]
Sep 11 18:18:11.255783 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:18:11.255756    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73faf9e3_65f8_4816_bdee_82b1e8a0227a.slice/crio-88d520049f0e0361cd3e58fd8c1c19ec8bf2549871589793da71890cff83401d WatchSource:0}: Error finding container 88d520049f0e0361cd3e58fd8c1c19ec8bf2549871589793da71890cff83401d: Status 404 returned error can't find the container with id 88d520049f0e0361cd3e58fd8c1c19ec8bf2549871589793da71890cff83401d
Sep 11 18:18:11.320959 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:11.320920    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68756467d4-gmxt5"]
Sep 11 18:18:11.323601 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:18:11.323571    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a936883_9636_4760_9260_27a0c9a7731b.slice/crio-a3c556b7cb9b91752a28a98fa9c9f972fb97b306bcabf8bb7cb4e4d3b0436bcb WatchSource:0}: Error finding container a3c556b7cb9b91752a28a98fa9c9f972fb97b306bcabf8bb7cb4e4d3b0436bcb: Status 404 returned error can't find the container with id a3c556b7cb9b91752a28a98fa9c9f972fb97b306bcabf8bb7cb4e4d3b0436bcb
Sep 11 18:18:12.012311 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:12.011984    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68756467d4-gmxt5" event={"ID":"5a936883-9636-4760-9260-27a0c9a7731b","Type":"ContainerStarted","Data":"3b9ec5fde38d918d6fc903029f97bcfbb77d6e485c7b8434246cb09c200ed3ca"}
Sep 11 18:18:12.012311 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:12.012046    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68756467d4-gmxt5" event={"ID":"5a936883-9636-4760-9260-27a0c9a7731b","Type":"ContainerStarted","Data":"a3c556b7cb9b91752a28a98fa9c9f972fb97b306bcabf8bb7cb4e4d3b0436bcb"}
Sep 11 18:18:12.012311 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:12.012091    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:12.013746 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:12.013622    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm" event={"ID":"73faf9e3-65f8-4816-bdee-82b1e8a0227a","Type":"ContainerStarted","Data":"88d520049f0e0361cd3e58fd8c1c19ec8bf2549871589793da71890cff83401d"}
Sep 11 18:18:12.039766 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:12.039228    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68756467d4-gmxt5" podStartSLOduration=2.039208548 podStartE2EDuration="2.039208548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:12.038980133 +0000 UTC m=+155.074196807" watchObservedRunningTime="2026-09-11 18:18:12.039208548 +0000 UTC m=+155.074425295"
Sep 11 18:18:14.022174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:14.022093    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm" event={"ID":"73faf9e3-65f8-4816-bdee-82b1e8a0227a","Type":"ContainerStarted","Data":"c3c8b51f91bbc1b8c546e8fdbb52abaa37833e0a3b76da422fcdc7135aa7389a"}
Sep 11 18:18:14.036298 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:14.036252    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dc495bb6-zrwbm" podStartSLOduration=1.579209826 podStartE2EDuration="4.036236808s" totalImagesPullingTime="2.457026982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:14.035713091 +0000 UTC m=+157.070929760" watchObservedRunningTime="2026-09-11 18:18:14.036236808 +0000 UTC m=+157.071453477"
Sep 11 18:18:14.403074 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:14.402967    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-88vkz" podUID="46bfe80b-cd69-4ba9-a7fb-128f20aa12d0"
Sep 11 18:18:15.026224 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:15.026194    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-88vkz"
Sep 11 18:18:15.427587 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:15.427544    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bht8j" podUID="c3d9bcff-2714-44ac-8bd8-e288668a8a7b"
Sep 11 18:18:16.374402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:16.374302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:16.375104 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:16.375079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1d12737-e1eb-4321-9473-e06391ac2642-service-ca-bundle\") pod \"router-default-7bf46bd65d-jtnbl\" (UID: \"f1d12737-e1eb-4321-9473-e06391ac2642\") " pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:16.581838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:16.581787    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:16.854841 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:16.854811    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7bf46bd65d-jtnbl"]
Sep 11 18:18:16.857737 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:18:16.857709    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d12737_e1eb_4321_9473_e06391ac2642.slice/crio-74c364fa26385ad09dcf4ca0bd3603b99f0b8a752c812d0edd519b8538b8ec85 WatchSource:0}: Error finding container 74c364fa26385ad09dcf4ca0bd3603b99f0b8a752c812d0edd519b8538b8ec85: Status 404 returned error can't find the container with id 74c364fa26385ad09dcf4ca0bd3603b99f0b8a752c812d0edd519b8538b8ec85
Sep 11 18:18:17.034614 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:17.034575    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7bf46bd65d-jtnbl" event={"ID":"f1d12737-e1eb-4321-9473-e06391ac2642","Type":"ContainerStarted","Data":"3e52d4cb4414c345f013e2050f97ef5a87fda2923ec387bbc326e1f012ddaad1"}
Sep 11 18:18:17.034793 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:17.034629    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7bf46bd65d-jtnbl" event={"ID":"f1d12737-e1eb-4321-9473-e06391ac2642","Type":"ContainerStarted","Data":"74c364fa26385ad09dcf4ca0bd3603b99f0b8a752c812d0edd519b8538b8ec85"}
Sep 11 18:18:17.055472 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:17.055417    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7bf46bd65d-jtnbl" podStartSLOduration=33.055401266 podStartE2EDuration="33.055401266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:17:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:17.054840996 +0000 UTC m=+160.090057666" watchObservedRunningTime="2026-09-11 18:18:17.055401266 +0000 UTC m=+160.090617935"
Sep 11 18:18:17.582148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:17.582112    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:17.584651 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:17.584624    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:18.038918 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:18.038884    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:18.040145 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:18.040122    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7bf46bd65d-jtnbl"
Sep 11 18:18:19.295963 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:19.295926    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:18:19.298217 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:19.298184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46bfe80b-cd69-4ba9-a7fb-128f20aa12d0-metrics-tls\") pod \"dns-default-88vkz\" (UID: \"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0\") " pod="openshift-dns/dns-default-88vkz"
Sep 11 18:18:19.529440 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:19.529408    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ksdw9\""
Sep 11 18:18:19.537113 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:19.537093    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-88vkz"
Sep 11 18:18:19.808091 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:19.808001    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-88vkz"]
Sep 11 18:18:19.810912 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:18:19.810886    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46bfe80b_cd69_4ba9_a7fb_128f20aa12d0.slice/crio-a3c4a9c6ecd1aeabf1ae9d2e4d0d9e87d2be44a6a8f809bfb4c6764ae3d35a96 WatchSource:0}: Error finding container a3c4a9c6ecd1aeabf1ae9d2e4d0d9e87d2be44a6a8f809bfb4c6764ae3d35a96: Status 404 returned error can't find the container with id a3c4a9c6ecd1aeabf1ae9d2e4d0d9e87d2be44a6a8f809bfb4c6764ae3d35a96
Sep 11 18:18:20.046313 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:20.046271    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-88vkz" event={"ID":"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0","Type":"ContainerStarted","Data":"a3c4a9c6ecd1aeabf1ae9d2e4d0d9e87d2be44a6a8f809bfb4c6764ae3d35a96"}
Sep 11 18:18:21.209548 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:21.209506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:18:21.212108 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:21.212081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bbc029c-d8f7-46bc-bfe9-a55922054b8b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmtmq\" (UID: \"9bbc029c-d8f7-46bc-bfe9-a55922054b8b\") " pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:18:21.413743 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:21.413719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hmtmq"
Sep 11 18:18:21.697825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:21.697799    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hmtmq"]
Sep 11 18:18:21.701394 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:18:21.701351    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bbc029c_d8f7_46bc_bfe9_a55922054b8b.slice/crio-b155c7d613749b06ab1840712f017393ff2545b553c3c0edfd7679fe0cf3b3c3 WatchSource:0}: Error finding container b155c7d613749b06ab1840712f017393ff2545b553c3c0edfd7679fe0cf3b3c3: Status 404 returned error can't find the container with id b155c7d613749b06ab1840712f017393ff2545b553c3c0edfd7679fe0cf3b3c3
Sep 11 18:18:22.055940 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:22.055848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-88vkz" event={"ID":"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0","Type":"ContainerStarted","Data":"c3be956c4aaff1cc7921d9753b6f7e9de080837acdd0819abd122a0eb8957e68"}
Sep 11 18:18:22.055940 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:22.055911    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-88vkz" event={"ID":"46bfe80b-cd69-4ba9-a7fb-128f20aa12d0","Type":"ContainerStarted","Data":"beb03c20260a71e0cbdfe9ce3d587f409b957f0b99317dcd13e66baecdd968f2"}
Sep 11 18:18:22.056195 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:22.056031    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-88vkz"
Sep 11 18:18:22.057410 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:22.057387    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hmtmq" event={"ID":"9bbc029c-d8f7-46bc-bfe9-a55922054b8b","Type":"ContainerStarted","Data":"cbd2da924830762077853bb530193e45ee0979c032d6555e8476ea2275087c98"}
Sep 11 18:18:22.057529 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:22.057435    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hmtmq" event={"ID":"9bbc029c-d8f7-46bc-bfe9-a55922054b8b","Type":"ContainerStarted","Data":"b155c7d613749b06ab1840712f017393ff2545b553c3c0edfd7679fe0cf3b3c3"}
Sep 11 18:18:22.072061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:22.072017    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-88vkz" podStartSLOduration=129.237415641 podStartE2EDuration="2m11.072001328s" totalImagesPullingTime="1.834585687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:22.071647426 +0000 UTC m=+165.106864096" watchObservedRunningTime="2026-09-11 18:18:22.072001328 +0000 UTC m=+165.107218001"
Sep 11 18:18:24.067373 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:24.067340    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hmtmq" event={"ID":"9bbc029c-d8f7-46bc-bfe9-a55922054b8b","Type":"ContainerStarted","Data":"a3d100c3f416da18b674835cf060e52e37bcbb5df04a7de9f32ac92250b1982f"}
Sep 11 18:18:25.072601 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:25.072561    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hmtmq" event={"ID":"9bbc029c-d8f7-46bc-bfe9-a55922054b8b","Type":"ContainerStarted","Data":"20c9628e4edb9bcb3f71f69340042ab05b1b63740c1b2e2ece5f8e9dd5dbe7d0"}
Sep 11 18:18:25.090631 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:25.090582    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hmtmq" podStartSLOduration=33.533638344 podStartE2EDuration="36.090568088s" totalImagesPullingTime="2.556929744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:17:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:25.090463155 +0000 UTC m=+168.125679826" watchObservedRunningTime="2026-09-11 18:18:25.090568088 +0000 UTC m=+168.125784758"
Sep 11 18:18:27.405285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:27.405240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:18:31.045532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:31.045449    2578 patch_prober.go:28] interesting pod/image-registry-68756467d4-gmxt5 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 18:18:31.045532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:31.045505    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-68756467d4-gmxt5" podUID="5a936883-9636-4760-9260-27a0c9a7731b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:18:32.064183 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:32.064148    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-88vkz"
Sep 11 18:18:33.022376 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:33.022345    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68756467d4-gmxt5"
Sep 11 18:18:36.510287 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.510249    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"]
Sep 11 18:18:36.512598 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.512573    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.516838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.516815    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 18:18:36.517821 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.517798    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 18:18:36.517821 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.517812    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dddz2\""
Sep 11 18:18:36.517970 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.517823    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 18:18:36.517970 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.517864    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 18:18:36.517970 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.517802    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 18:18:36.525217 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.525196    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"]
Sep 11 18:18:36.533075 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.533055    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fbdpb"]
Sep 11 18:18:36.535043 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.535025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.537399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.537381    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 18:18:36.537503 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.537469    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 18:18:36.537503 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.537482    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kzmdz\""
Sep 11 18:18:36.537619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.537556    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 18:18:36.543976 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.543956    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"]
Sep 11 18:18:36.545906 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.545893    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.549057 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.549033    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 11 18:18:36.549155 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.549091    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 11 18:18:36.549390 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.549374    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 11 18:18:36.549472 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.549390    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rrhm8\""
Sep 11 18:18:36.565614 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.565592    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"]
Sep 11 18:18:36.632706 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.632661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a69938cd-b755-49e7-85a1-e3775324d382-sys\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.632706 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.632709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.632912 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.632750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpnhb\" (UniqueName: \"kubernetes.io/projected/a69938cd-b755-49e7-85a1-e3775324d382-kube-api-access-zpnhb\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.632912 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.632769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.632912 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.632786    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.632912 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.632809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c258\" (UniqueName: \"kubernetes.io/projected/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-api-access-9c258\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.632912 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.632871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-textfile\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.632912 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.632904    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7089dec3-ca65-4924-9104-e5029434d860-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.633094 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.632973    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7089dec3-ca65-4924-9104-e5029434d860-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.633094 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633016    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37e3a74c-b152-456d-a962-3785e8f5ed6d-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.633094 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a69938cd-b755-49e7-85a1-e3775324d382-metrics-client-ca\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.633094 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7089dec3-ca65-4924-9104-e5029434d860-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.633094 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-wtmp\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.633270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633138    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a69938cd-b755-49e7-85a1-e3775324d382-root\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.633270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srt8d\" (UniqueName: \"kubernetes.io/projected/7089dec3-ca65-4924-9104-e5029434d860-kube-api-access-srt8d\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.633270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-accelerators-collector-config\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.633270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633222    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-tls\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.633270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633237    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.633270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.633254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37e3a74c-b152-456d-a962-3785e8f5ed6d-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.734385 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.734385 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734385    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpnhb\" (UniqueName: \"kubernetes.io/projected/a69938cd-b755-49e7-85a1-e3775324d382-kube-api-access-zpnhb\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c258\" (UniqueName: \"kubernetes.io/projected/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-api-access-9c258\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734488    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-textfile\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734511    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7089dec3-ca65-4924-9104-e5029434d860-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:36.734525    2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7089dec3-ca65-4924-9104-e5029434d860-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37e3a74c-b152-456d-a962-3785e8f5ed6d-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:36.734589    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-tls podName:37e3a74c-b152-456d-a962-3785e8f5ed6d nodeName:}" failed. No retries permitted until 2026-09-11 18:18:37.234568336 +0000 UTC m=+180.269785004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-8crdq" (UID: "37e3a74c-b152-456d-a962-3785e8f5ed6d") : secret "kube-state-metrics-tls" not found
Sep 11 18:18:36.734612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a69938cd-b755-49e7-85a1-e3775324d382-metrics-client-ca\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734647    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7089dec3-ca65-4924-9104-e5029434d860-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-wtmp\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a69938cd-b755-49e7-85a1-e3775324d382-root\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734799    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srt8d\" (UniqueName: \"kubernetes.io/projected/7089dec3-ca65-4924-9104-e5029434d860-kube-api-access-srt8d\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734843    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-accelerators-collector-config\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734855    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-textfile\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734875    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-tls\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37e3a74c-b152-456d-a962-3785e8f5ed6d-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:36.734956    2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734991    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37e3a74c-b152-456d-a962-3785e8f5ed6d-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.735006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a69938cd-b755-49e7-85a1-e3775324d382-sys\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:36.735023    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7089dec3-ca65-4924-9104-e5029434d860-openshift-state-metrics-tls podName:7089dec3-ca65-4924-9104-e5029434d860 nodeName:}" failed. No retries permitted until 2026-09-11 18:18:37.235005584 +0000 UTC m=+180.270222244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7089dec3-ca65-4924-9104-e5029434d860-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-gsjvb" (UID: "7089dec3-ca65-4924-9104-e5029434d860") : secret "openshift-state-metrics-tls" not found
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.734962    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a69938cd-b755-49e7-85a1-e3775324d382-sys\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.735120    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a69938cd-b755-49e7-85a1-e3775324d382-root\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.735141    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-wtmp\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735992 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.735229    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a69938cd-b755-49e7-85a1-e3775324d382-metrics-client-ca\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735992 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:36.735312    2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 18:18:36.735992 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:18:36.735368    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-tls podName:a69938cd-b755-49e7-85a1-e3775324d382 nodeName:}" failed. No retries permitted until 2026-09-11 18:18:37.235350705 +0000 UTC m=+180.270567370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-tls") pod "node-exporter-fbdpb" (UID: "a69938cd-b755-49e7-85a1-e3775324d382") : secret "node-exporter-tls" not found
Sep 11 18:18:36.735992 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.735417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7089dec3-ca65-4924-9104-e5029434d860-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.735992 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.735773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37e3a74c-b152-456d-a962-3785e8f5ed6d-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.735992 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.735887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-accelerators-collector-config\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.735992 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.735943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.737215 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.737184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.737334 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.737298    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7089dec3-ca65-4924-9104-e5029434d860-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.737440 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.737420    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:36.745416 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.745385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c258\" (UniqueName: \"kubernetes.io/projected/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-api-access-9c258\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:36.745540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.745517    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srt8d\" (UniqueName: \"kubernetes.io/projected/7089dec3-ca65-4924-9104-e5029434d860-kube-api-access-srt8d\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:36.745782 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:36.745766    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpnhb\" (UniqueName: \"kubernetes.io/projected/a69938cd-b755-49e7-85a1-e3775324d382-kube-api-access-zpnhb\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:37.239265 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.239228    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7089dec3-ca65-4924-9104-e5029434d860-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:37.239473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.239412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-tls\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:37.239542 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.239477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:37.241670 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.241645    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69938cd-b755-49e7-85a1-e3775324d382-node-exporter-tls\") pod \"node-exporter-fbdpb\" (UID: \"a69938cd-b755-49e7-85a1-e3775324d382\") " pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:37.241812 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.241724    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7089dec3-ca65-4924-9104-e5029434d860-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-gsjvb\" (UID: \"7089dec3-ca65-4924-9104-e5029434d860\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:37.241812 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.241747    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37e3a74c-b152-456d-a962-3785e8f5ed6d-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-8crdq\" (UID: \"37e3a74c-b152-456d-a962-3785e8f5ed6d\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:37.422939 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.422910    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dddz2\""
Sep 11 18:18:37.431633 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.431604    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"
Sep 11 18:18:37.446559 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.446534    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kzmdz\""
Sep 11 18:18:37.454932 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.454910    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fbdpb"
Sep 11 18:18:37.455203 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.455179    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rrhm8\""
Sep 11 18:18:37.464082 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.464052    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"
Sep 11 18:18:37.467165 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:18:37.467136    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69938cd_b755_49e7_85a1_e3775324d382.slice/crio-847dfac7a58a32466f421164d52c1fc2bd50b54b84c961360715438237c3cd13 WatchSource:0}: Error finding container 847dfac7a58a32466f421164d52c1fc2bd50b54b84c961360715438237c3cd13: Status 404 returned error can't find the container with id 847dfac7a58a32466f421164d52c1fc2bd50b54b84c961360715438237c3cd13
Sep 11 18:18:37.704613 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.704586    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb"]
Sep 11 18:18:37.707474 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:18:37.707452    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7089dec3_ca65_4924_9104_e5029434d860.slice/crio-7f0be24e19f88b0f7e0be74ddf7a9cc1896133e325215f1c796dbd3850f9246b WatchSource:0}: Error finding container 7f0be24e19f88b0f7e0be74ddf7a9cc1896133e325215f1c796dbd3850f9246b: Status 404 returned error can't find the container with id 7f0be24e19f88b0f7e0be74ddf7a9cc1896133e325215f1c796dbd3850f9246b
Sep 11 18:18:37.743931 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:37.743528    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq"]
Sep 11 18:18:37.748818 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:18:37.748789    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e3a74c_b152_456d_a962_3785e8f5ed6d.slice/crio-9069e6271a6a8ed4c072bf94b8952ec2507e7adcf4b92b32f1330c07a38d821b WatchSource:0}: Error finding container 9069e6271a6a8ed4c072bf94b8952ec2507e7adcf4b92b32f1330c07a38d821b: Status 404 returned error can't find the container with id 9069e6271a6a8ed4c072bf94b8952ec2507e7adcf4b92b32f1330c07a38d821b
Sep 11 18:18:38.115810 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:38.115719    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq" event={"ID":"37e3a74c-b152-456d-a962-3785e8f5ed6d","Type":"ContainerStarted","Data":"9069e6271a6a8ed4c072bf94b8952ec2507e7adcf4b92b32f1330c07a38d821b"}
Sep 11 18:18:38.117731 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:38.117703    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb" event={"ID":"7089dec3-ca65-4924-9104-e5029434d860","Type":"ContainerStarted","Data":"1c0e958beb4c785a43293c070c04c05a99bb300232dbecf88c30afd114eec149"}
Sep 11 18:18:38.117865 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:38.117773    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb" event={"ID":"7089dec3-ca65-4924-9104-e5029434d860","Type":"ContainerStarted","Data":"bac6c87d09136b827bdc6f35329b5d779e0bd6a7729757c7ec85bb5b7620bddf"}
Sep 11 18:18:38.117865 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:38.117801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb" event={"ID":"7089dec3-ca65-4924-9104-e5029434d860","Type":"ContainerStarted","Data":"7f0be24e19f88b0f7e0be74ddf7a9cc1896133e325215f1c796dbd3850f9246b"}
Sep 11 18:18:38.119040 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:38.119019    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbdpb" event={"ID":"a69938cd-b755-49e7-85a1-e3775324d382","Type":"ContainerStarted","Data":"847dfac7a58a32466f421164d52c1fc2bd50b54b84c961360715438237c3cd13"}
Sep 11 18:18:39.134114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:39.134081    2578 generic.go:356] "Generic (PLEG): container finished" podID="a69938cd-b755-49e7-85a1-e3775324d382" containerID="09f26c0d65406068a3b4285fd861bbded735d541a76a58e09852b06d8207ab62" exitCode=0
Sep 11 18:18:39.134428 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:39.134140    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbdpb" event={"ID":"a69938cd-b755-49e7-85a1-e3775324d382","Type":"ContainerDied","Data":"09f26c0d65406068a3b4285fd861bbded735d541a76a58e09852b06d8207ab62"}
Sep 11 18:18:40.139521 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:40.139486    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbdpb" event={"ID":"a69938cd-b755-49e7-85a1-e3775324d382","Type":"ContainerStarted","Data":"ae6d9e23dbaa958493e06fe0bd4895f5842a44a98e2476a96d716cd24a25f3de"}
Sep 11 18:18:40.139968 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:40.139540    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbdpb" event={"ID":"a69938cd-b755-49e7-85a1-e3775324d382","Type":"ContainerStarted","Data":"36b5a944cd0f2442f3de6396e626e8348bd0ac44a124486ac5d9a857e103efd3"}
Sep 11 18:18:40.141673 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:40.141646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq" event={"ID":"37e3a74c-b152-456d-a962-3785e8f5ed6d","Type":"ContainerStarted","Data":"c7ed741e6578299a926572c3c6f5dded10f4d4e4bebd2b2f84fe6d7141c1a346"}
Sep 11 18:18:40.141816 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:40.141718    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq" event={"ID":"37e3a74c-b152-456d-a962-3785e8f5ed6d","Type":"ContainerStarted","Data":"f82632e6d0be0bf3975d410bd09362d25a1d456d24e52eb12af39fe27f60f3ce"}
Sep 11 18:18:40.141816 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:40.141749    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq" event={"ID":"37e3a74c-b152-456d-a962-3785e8f5ed6d","Type":"ContainerStarted","Data":"f2b80a2be046c1bbfbe1d65e34851325713379e03701f010d805001d6183eb91"}
Sep 11 18:18:40.143588 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:40.143567    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb" event={"ID":"7089dec3-ca65-4924-9104-e5029434d860","Type":"ContainerStarted","Data":"3fcf747997837c789c4f35b5a83c7c50e51048f213f8f36341202ff4abe849a9"}
Sep 11 18:18:40.161628 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:40.161588    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fbdpb" podStartSLOduration=3.118718093 podStartE2EDuration="4.161577s" totalImagesPullingTime="963.540246ms" totalInitContainerRuntime="79.318661ms" isStatefulPod=true podCreationTimestamp="2026-09-11 18:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:40.161149343 +0000 UTC m=+183.196366013" watchObservedRunningTime="2026-09-11 18:18:40.161577 +0000 UTC m=+183.196793667"
Sep 11 18:18:40.178612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:40.178569    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-gsjvb" podStartSLOduration=2.91930047 podStartE2EDuration="4.178556066s" totalImagesPullingTime="1.259255596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:40.177605622 +0000 UTC m=+183.212822293" watchObservedRunningTime="2026-09-11 18:18:40.178556066 +0000 UTC m=+183.213772736"
Sep 11 18:18:40.197344 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:40.197298    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8crdq" podStartSLOduration=2.839779138 podStartE2EDuration="4.197281051s" totalImagesPullingTime="1.357501913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:40.197175178 +0000 UTC m=+183.232391849" watchObservedRunningTime="2026-09-11 18:18:40.197281051 +0000 UTC m=+183.232497723"
Sep 11 18:18:41.244570 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.244538    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"]
Sep 11 18:18:41.247476 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.247453    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:41.250183 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.250165    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hdd74\""
Sep 11 18:18:41.250300 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.250281    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 11 18:18:41.255425 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.255404    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"]
Sep 11 18:18:41.375839 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.375808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z24t7\" (UniqueName: \"kubernetes.io/projected/67e25494-876a-44de-bc40-8feaabc007c2-kube-api-access-z24t7\") pod \"monitoring-plugin-5d85778b9d-bxnnd\" (UID: \"67e25494-876a-44de-bc40-8feaabc007c2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:41.375955 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.375849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67e25494-876a-44de-bc40-8feaabc007c2-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-bxnnd\" (UID: \"67e25494-876a-44de-bc40-8feaabc007c2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:41.476733 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.476694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z24t7\" (UniqueName: \"kubernetes.io/projected/67e25494-876a-44de-bc40-8feaabc007c2-kube-api-access-z24t7\") pod \"monitoring-plugin-5d85778b9d-bxnnd\" (UID: \"67e25494-876a-44de-bc40-8feaabc007c2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:41.476908 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.476748    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67e25494-876a-44de-bc40-8feaabc007c2-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-bxnnd\" (UID: \"67e25494-876a-44de-bc40-8feaabc007c2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:41.479118 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.479096    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67e25494-876a-44de-bc40-8feaabc007c2-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-bxnnd\" (UID: \"67e25494-876a-44de-bc40-8feaabc007c2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:41.485123 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.485100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z24t7\" (UniqueName: \"kubernetes.io/projected/67e25494-876a-44de-bc40-8feaabc007c2-kube-api-access-z24t7\") pod \"monitoring-plugin-5d85778b9d-bxnnd\" (UID: \"67e25494-876a-44de-bc40-8feaabc007c2\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:41.557339 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.557284    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:41.824081 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:41.823990    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"]
Sep 11 18:18:41.826724 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:18:41.826697    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e25494_876a_44de_bc40_8feaabc007c2.slice/crio-1befbd462be20a3980764e9b47f05027b19a9569d8e641cc587cc4400198588f WatchSource:0}: Error finding container 1befbd462be20a3980764e9b47f05027b19a9569d8e641cc587cc4400198588f: Status 404 returned error can't find the container with id 1befbd462be20a3980764e9b47f05027b19a9569d8e641cc587cc4400198588f
Sep 11 18:18:42.158388 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:42.158356    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd" event={"ID":"67e25494-876a-44de-bc40-8feaabc007c2","Type":"ContainerStarted","Data":"1befbd462be20a3980764e9b47f05027b19a9569d8e641cc587cc4400198588f"}
Sep 11 18:18:43.166122 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:43.166088    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd" event={"ID":"67e25494-876a-44de-bc40-8feaabc007c2","Type":"ContainerStarted","Data":"a609cb7f8b81f04249864bd3ba67d99024d6d24141e4f0e895888f374956a4cd"}
Sep 11 18:18:43.166586 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:43.166219    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:43.170876 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:43.170853    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd"
Sep 11 18:18:43.181538 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:18:43.181501    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-bxnnd" podStartSLOduration=1.073471596 podStartE2EDuration="2.18149005s" totalImagesPullingTime="1.108018454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:43.180540944 +0000 UTC m=+186.215757616" watchObservedRunningTime="2026-09-11 18:18:43.18149005 +0000 UTC m=+186.216706719"
Sep 11 18:19:04.240002 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:04.239967    2578 generic.go:356] "Generic (PLEG): container finished" podID="818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7" containerID="c4c1c6c257f480dd43f9b5a0f315017a52d280b9c426bdfce59652891ac789fb" exitCode=0
Sep 11 18:19:04.240526 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:04.240047    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw" event={"ID":"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7","Type":"ContainerDied","Data":"c4c1c6c257f480dd43f9b5a0f315017a52d280b9c426bdfce59652891ac789fb"}
Sep 11 18:19:04.240526 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:04.240458    2578 scope.go:117] "RemoveContainer" containerID="c4c1c6c257f480dd43f9b5a0f315017a52d280b9c426bdfce59652891ac789fb"
Sep 11 18:19:05.245821 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:05.245783    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-cr7cw" event={"ID":"818f3ffd-a8f8-4a3d-b234-b330a5fd8cc7","Type":"ContainerStarted","Data":"8d56e3551012b3db1f312b2216b91e68079a205ece78dea9cd87fafcbdefe78d"}
Sep 11 18:19:14.277883 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:14.277850    2578 generic.go:356] "Generic (PLEG): container finished" podID="566a7849-1b81-4d2d-b3ad-80cc5d8031dc" containerID="58914f934a31d02b98678349b248f3b065012b07883014e3c8bd1fbe16ee7b86" exitCode=0
Sep 11 18:19:14.278364 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:14.277890    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-dbst7" event={"ID":"566a7849-1b81-4d2d-b3ad-80cc5d8031dc","Type":"ContainerDied","Data":"58914f934a31d02b98678349b248f3b065012b07883014e3c8bd1fbe16ee7b86"}
Sep 11 18:19:14.278364 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:14.278304    2578 scope.go:117] "RemoveContainer" containerID="58914f934a31d02b98678349b248f3b065012b07883014e3c8bd1fbe16ee7b86"
Sep 11 18:19:15.282630 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:15.282595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-dbst7" event={"ID":"566a7849-1b81-4d2d-b3ad-80cc5d8031dc","Type":"ContainerStarted","Data":"3c4e55757392ef5ed8a2e8878ae3e73211097e41d636c79053e62ee54261b7b3"}
Sep 11 18:19:49.222771 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:49.222737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:19:49.225037 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:49.225017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3d9bcff-2714-44ac-8bd8-e288668a8a7b-metrics-certs\") pod \"network-metrics-daemon-bht8j\" (UID: \"c3d9bcff-2714-44ac-8bd8-e288668a8a7b\") " pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:19:49.308731 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:49.308703    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kq7d5\""
Sep 11 18:19:49.316524 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:49.316496    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bht8j"
Sep 11 18:19:49.587639 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:49.587562    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bht8j"]
Sep 11 18:19:49.591330 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:19:49.591304    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d9bcff_2714_44ac_8bd8_e288668a8a7b.slice/crio-7263b8d270a28db95a78f73e17b544bbeba6fdcb8e64adc00a72053bb3f4e1dd WatchSource:0}: Error finding container 7263b8d270a28db95a78f73e17b544bbeba6fdcb8e64adc00a72053bb3f4e1dd: Status 404 returned error can't find the container with id 7263b8d270a28db95a78f73e17b544bbeba6fdcb8e64adc00a72053bb3f4e1dd
Sep 11 18:19:50.409658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:50.409616    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bht8j" event={"ID":"c3d9bcff-2714-44ac-8bd8-e288668a8a7b","Type":"ContainerStarted","Data":"7263b8d270a28db95a78f73e17b544bbeba6fdcb8e64adc00a72053bb3f4e1dd"}
Sep 11 18:19:51.414515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:51.414477    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bht8j" event={"ID":"c3d9bcff-2714-44ac-8bd8-e288668a8a7b","Type":"ContainerStarted","Data":"4f7ca9dbe7d19968cde24284aa7f4fab8f93a274d3bcfcaeaf5384d0bb1ed3c7"}
Sep 11 18:19:51.415010 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:51.414537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bht8j" event={"ID":"c3d9bcff-2714-44ac-8bd8-e288668a8a7b","Type":"ContainerStarted","Data":"a3df1dea7d9b33b2fc4a965d2cd2aa22762ab53f79a7644fa137cf3dd6516214"}
Sep 11 18:19:51.430366 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:19:51.430319    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bht8j" podStartSLOduration=252.47702393 podStartE2EDuration="4m13.430303051s" totalImagesPullingTime="953.279121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:19:51.428361092 +0000 UTC m=+254.463577761" watchObservedRunningTime="2026-09-11 18:19:51.430303051 +0000 UTC m=+254.465519792"
Sep 11 18:20:37.345661 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:20:37.345629    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:20:37.348167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:20:37.348136    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:20:37.349800 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:20:37.349781    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:20:37.352902 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:20:37.352878    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:20:37.357595 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:20:37.357579    2578 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 18:22:31.048618 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.048584    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"]
Sep 11 18:22:31.052123 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.052105    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.057101 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.057077    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lgxhw\""
Sep 11 18:22:31.057255 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.057162    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 18:22:31.057334 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.057268    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 18:22:31.068941 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.068919    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"]
Sep 11 18:22:31.130978 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.130953    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffdfd\" (UniqueName: \"kubernetes.io/projected/0dbf0fac-d3ed-46eb-b125-e906b95c951c-kube-api-access-ffdfd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.131111 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.131000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.131111 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.131027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.232255 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.232229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffdfd\" (UniqueName: \"kubernetes.io/projected/0dbf0fac-d3ed-46eb-b125-e906b95c951c-kube-api-access-ffdfd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.232377 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.232278    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.232377 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.232307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.232712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.232665    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.232820 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.232721    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.249132 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.249101    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffdfd\" (UniqueName: \"kubernetes.io/projected/0dbf0fac-d3ed-46eb-b125-e906b95c951c-kube-api-access-ffdfd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.360174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.360107    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:31.639399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.639374    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"]
Sep 11 18:22:31.641531 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:22:31.641500    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbf0fac_d3ed_46eb_b125_e906b95c951c.slice/crio-5e4c4ec967faf75a3d2bda21481a3936ed0578bc8e86dd7c9861d76bfd9a88ef WatchSource:0}: Error finding container 5e4c4ec967faf75a3d2bda21481a3936ed0578bc8e86dd7c9861d76bfd9a88ef: Status 404 returned error can't find the container with id 5e4c4ec967faf75a3d2bda21481a3936ed0578bc8e86dd7c9861d76bfd9a88ef
Sep 11 18:22:31.643433 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.643416    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:22:31.997515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:31.997481    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95" event={"ID":"0dbf0fac-d3ed-46eb-b125-e906b95c951c","Type":"ContainerStarted","Data":"5e4c4ec967faf75a3d2bda21481a3936ed0578bc8e86dd7c9861d76bfd9a88ef"}
Sep 11 18:22:39.027289 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:39.027254    2578 generic.go:356] "Generic (PLEG): container finished" podID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerID="da3e4bea7cb11a681c6547f82eeaa9930f22ad9dda13a1f64e192a4d642712b5" exitCode=0
Sep 11 18:22:39.027810 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:39.027354    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95" event={"ID":"0dbf0fac-d3ed-46eb-b125-e906b95c951c","Type":"ContainerDied","Data":"da3e4bea7cb11a681c6547f82eeaa9930f22ad9dda13a1f64e192a4d642712b5"}
Sep 11 18:22:41.039011 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:41.038983    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95" event={"ID":"0dbf0fac-d3ed-46eb-b125-e906b95c951c","Type":"ContainerStarted","Data":"f70a82c448207577707ab7be5e18e92897a00c3a5ede8d46f64edba3edf7431c"}
Sep 11 18:22:42.044744 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:42.044709    2578 generic.go:356] "Generic (PLEG): container finished" podID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerID="f70a82c448207577707ab7be5e18e92897a00c3a5ede8d46f64edba3edf7431c" exitCode=0
Sep 11 18:22:42.045144 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:42.044796    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95" event={"ID":"0dbf0fac-d3ed-46eb-b125-e906b95c951c","Type":"ContainerDied","Data":"f70a82c448207577707ab7be5e18e92897a00c3a5ede8d46f64edba3edf7431c"}
Sep 11 18:22:49.077952 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:49.077873    2578 generic.go:356] "Generic (PLEG): container finished" podID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerID="a3c6d86aba7770eea2a39f1e0d9b4fba79169078e6771d259c85b99dde808f20" exitCode=0
Sep 11 18:22:49.078293 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:49.077956    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95" event={"ID":"0dbf0fac-d3ed-46eb-b125-e906b95c951c","Type":"ContainerDied","Data":"a3c6d86aba7770eea2a39f1e0d9b4fba79169078e6771d259c85b99dde808f20"}
Sep 11 18:22:50.200203 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.200182    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:50.281047 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.281018    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffdfd\" (UniqueName: \"kubernetes.io/projected/0dbf0fac-d3ed-46eb-b125-e906b95c951c-kube-api-access-ffdfd\") pod \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") "
Sep 11 18:22:50.283167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.283141    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dbf0fac-d3ed-46eb-b125-e906b95c951c-kube-api-access-ffdfd" (OuterVolumeSpecName: "kube-api-access-ffdfd") pod "0dbf0fac-d3ed-46eb-b125-e906b95c951c" (UID: "0dbf0fac-d3ed-46eb-b125-e906b95c951c"). InnerVolumeSpecName "kube-api-access-ffdfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:22:50.382367 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.382287    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-util\") pod \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") "
Sep 11 18:22:50.382367 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.382338    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-bundle\") pod \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\" (UID: \"0dbf0fac-d3ed-46eb-b125-e906b95c951c\") "
Sep 11 18:22:50.382525 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.382437    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffdfd\" (UniqueName: \"kubernetes.io/projected/0dbf0fac-d3ed-46eb-b125-e906b95c951c-kube-api-access-ffdfd\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:22:50.382732 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.382709    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-bundle" (OuterVolumeSpecName: "bundle") pod "0dbf0fac-d3ed-46eb-b125-e906b95c951c" (UID: "0dbf0fac-d3ed-46eb-b125-e906b95c951c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:22:50.386169 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.386150    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-util" (OuterVolumeSpecName: "util") pod "0dbf0fac-d3ed-46eb-b125-e906b95c951c" (UID: "0dbf0fac-d3ed-46eb-b125-e906b95c951c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:22:50.483210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.483172    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-util\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:22:50.483210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:50.483207    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0dbf0fac-d3ed-46eb-b125-e906b95c951c-bundle\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:22:51.087047 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:51.087014    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95" event={"ID":"0dbf0fac-d3ed-46eb-b125-e906b95c951c","Type":"ContainerDied","Data":"5e4c4ec967faf75a3d2bda21481a3936ed0578bc8e86dd7c9861d76bfd9a88ef"}
Sep 11 18:22:51.087047 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:51.087040    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg6z95"
Sep 11 18:22:51.087264 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:51.087060    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e4c4ec967faf75a3d2bda21481a3936ed0578bc8e86dd7c9861d76bfd9a88ef"
Sep 11 18:22:56.032660 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.032622    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"]
Sep 11 18:22:56.033048 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.032901    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerName="extract"
Sep 11 18:22:56.033048 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.032912    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerName="extract"
Sep 11 18:22:56.033048 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.032922    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerName="util"
Sep 11 18:22:56.033048 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.032928    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerName="util"
Sep 11 18:22:56.033048 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.032942    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerName="pull"
Sep 11 18:22:56.033048 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.032947    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerName="pull"
Sep 11 18:22:56.033048 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.032995    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbf0fac-d3ed-46eb-b125-e906b95c951c" containerName="extract"
Sep 11 18:22:56.069738 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.069704    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"]
Sep 11 18:22:56.069873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.069807    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"
Sep 11 18:22:56.072926 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.072904    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-c6mgg\""
Sep 11 18:22:56.072926 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.072918    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:22:56.073392 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.073375    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 18:22:56.126165 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.126137    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24g96\" (UniqueName: \"kubernetes.io/projected/21596dda-2c5c-4160-b93b-1d26af60194c-kube-api-access-24g96\") pod \"openshift-lws-operator-fd8ccff4c-8kcqf\" (UID: \"21596dda-2c5c-4160-b93b-1d26af60194c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"
Sep 11 18:22:56.126290 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.126186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21596dda-2c5c-4160-b93b-1d26af60194c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8kcqf\" (UID: \"21596dda-2c5c-4160-b93b-1d26af60194c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"
Sep 11 18:22:56.227572 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.227532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24g96\" (UniqueName: \"kubernetes.io/projected/21596dda-2c5c-4160-b93b-1d26af60194c-kube-api-access-24g96\") pod \"openshift-lws-operator-fd8ccff4c-8kcqf\" (UID: \"21596dda-2c5c-4160-b93b-1d26af60194c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"
Sep 11 18:22:56.227756 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.227600    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21596dda-2c5c-4160-b93b-1d26af60194c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8kcqf\" (UID: \"21596dda-2c5c-4160-b93b-1d26af60194c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"
Sep 11 18:22:56.228035 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.228014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21596dda-2c5c-4160-b93b-1d26af60194c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8kcqf\" (UID: \"21596dda-2c5c-4160-b93b-1d26af60194c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"
Sep 11 18:22:56.235947 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.235920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24g96\" (UniqueName: \"kubernetes.io/projected/21596dda-2c5c-4160-b93b-1d26af60194c-kube-api-access-24g96\") pod \"openshift-lws-operator-fd8ccff4c-8kcqf\" (UID: \"21596dda-2c5c-4160-b93b-1d26af60194c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"
Sep 11 18:22:56.378658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.378575    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"
Sep 11 18:22:56.673229 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:56.673199    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf"]
Sep 11 18:22:56.676918 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:22:56.676892    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21596dda_2c5c_4160_b93b_1d26af60194c.slice/crio-1362b09afa69b40aef9ca8ee63e186948141a13cac1dd71e09660a21b9e34a50 WatchSource:0}: Error finding container 1362b09afa69b40aef9ca8ee63e186948141a13cac1dd71e09660a21b9e34a50: Status 404 returned error can't find the container with id 1362b09afa69b40aef9ca8ee63e186948141a13cac1dd71e09660a21b9e34a50
Sep 11 18:22:57.110852 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:22:57.110768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf" event={"ID":"21596dda-2c5c-4160-b93b-1d26af60194c","Type":"ContainerStarted","Data":"1362b09afa69b40aef9ca8ee63e186948141a13cac1dd71e09660a21b9e34a50"}
Sep 11 18:23:00.124250 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:00.124212    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf" event={"ID":"21596dda-2c5c-4160-b93b-1d26af60194c","Type":"ContainerStarted","Data":"4cce481e69a385cbfe1cf3cd6e75319dee342a5916eb85fbbd86093a02294b96"}
Sep 11 18:23:00.148427 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:00.148372    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kcqf" podStartSLOduration=1.241465874 podStartE2EDuration="4.148356234s" totalImagesPullingTime="2.90689036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:00.147613662 +0000 UTC m=+443.182830333" watchObservedRunningTime="2026-09-11 18:23:00.148356234 +0000 UTC m=+443.183572903"
Sep 11 18:23:19.123368 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.123327    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"]
Sep 11 18:23:19.145239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.145204    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.145906 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.145880    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"]
Sep 11 18:23:19.152082 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.152054    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 18:23:19.152215 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.152084    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 18:23:19.152215 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.152086    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rqg22\""
Sep 11 18:23:19.152333 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.152093    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 18:23:19.207670 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.207643    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-manager-config\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.207820 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.207706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-metrics-cert\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.207820 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.207725    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-cert\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.207820 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.207751    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4c97\" (UniqueName: \"kubernetes.io/projected/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-kube-api-access-w4c97\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.308780 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.308752    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-manager-config\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.308903 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.308790    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-metrics-cert\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.308903 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.308809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-cert\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.308974 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.308927    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4c97\" (UniqueName: \"kubernetes.io/projected/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-kube-api-access-w4c97\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.309487 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.309462    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-manager-config\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.311380 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.311351    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-cert\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.311465 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.311351    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-metrics-cert\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.319282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.319260    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4c97\" (UniqueName: \"kubernetes.io/projected/cb3c6a8c-8ef7-4d06-b178-cdee0a263da1-kube-api-access-w4c97\") pod \"lws-controller-manager-78b8f7c5cb-zcqqs\" (UID: \"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1\") " pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.455219 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.455192    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:19.774450 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.774422    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"]
Sep 11 18:23:19.776856 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:23:19.776829    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3c6a8c_8ef7_4d06_b178_cdee0a263da1.slice/crio-6c5c9176bbc4b5d43ce9f9e10a477e69b9b7516a8f2b24fd1464b33bfca3a337 WatchSource:0}: Error finding container 6c5c9176bbc4b5d43ce9f9e10a477e69b9b7516a8f2b24fd1464b33bfca3a337: Status 404 returned error can't find the container with id 6c5c9176bbc4b5d43ce9f9e10a477e69b9b7516a8f2b24fd1464b33bfca3a337
Sep 11 18:23:19.864032 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.864003    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"]
Sep 11 18:23:19.866967 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.866947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:19.871119 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.871097    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 18:23:19.871249 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.871160    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q6wtr\""
Sep 11 18:23:19.871249 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.871102    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 18:23:19.871249 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.871187    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 18:23:19.871249 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.871238    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 18:23:19.896220 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.896195    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"]
Sep 11 18:23:19.914764 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.914737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxf2h\" (UniqueName: \"kubernetes.io/projected/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-kube-api-access-kxf2h\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:19.914880 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.914781    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:19.914880 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.914806    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-tmp\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:19.914988 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.914893    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:19.914988 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:19.914956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-manifests\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.015776 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.015738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-manifests\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.015951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.015792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxf2h\" (UniqueName: \"kubernetes.io/projected/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-kube-api-access-kxf2h\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.015951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.015820    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.015951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.015843    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-tmp\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.015951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.015866    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.016192 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.016171    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-manifests\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.018179 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.018156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-tmp\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.018318 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.018298    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.018385 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.018368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.024563 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.024517    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxf2h\" (UniqueName: \"kubernetes.io/projected/ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b-kube-api-access-kxf2h\") pod \"opendatahub-operator-controller-manager-68b7b6b756-c9p8m\" (UID: \"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b\") " pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.177568 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.177532    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:20.204818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.204789    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs" event={"ID":"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1","Type":"ContainerStarted","Data":"6c5c9176bbc4b5d43ce9f9e10a477e69b9b7516a8f2b24fd1464b33bfca3a337"}
Sep 11 18:23:20.480254 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:20.480216    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"]
Sep 11 18:23:20.483532 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:23:20.483500    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5b00f6_bd4a_405a_9c6f_4e296fa02c2b.slice/crio-20d584a29f49a120d95b8da17feb899bcd348f02b097c12c5f63d41a520a0f32 WatchSource:0}: Error finding container 20d584a29f49a120d95b8da17feb899bcd348f02b097c12c5f63d41a520a0f32: Status 404 returned error can't find the container with id 20d584a29f49a120d95b8da17feb899bcd348f02b097c12c5f63d41a520a0f32
Sep 11 18:23:21.212496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:21.212411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs" event={"ID":"cb3c6a8c-8ef7-4d06-b178-cdee0a263da1","Type":"ContainerStarted","Data":"bd82742b3d341e6a031f87fc35ac6b420d42b0c4332db48958a875814e3a9509"}
Sep 11 18:23:21.212958 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:21.212501    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:21.214359 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:21.214291    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m" event={"ID":"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b","Type":"ContainerStarted","Data":"20d584a29f49a120d95b8da17feb899bcd348f02b097c12c5f63d41a520a0f32"}
Sep 11 18:23:21.231840 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:21.231647    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs" podStartSLOduration=0.94599045 podStartE2EDuration="2.23162586s" totalImagesPullingTime="1.28563541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:21.23118671 +0000 UTC m=+464.266403392" watchObservedRunningTime="2026-09-11 18:23:21.23162586 +0000 UTC m=+464.266842530"
Sep 11 18:23:24.229488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:24.229444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m" event={"ID":"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b","Type":"ContainerStarted","Data":"cf8e5eb26023ebf64a38a26815f72753cc6483ffcaa28bb1bf874081f125e51f"}
Sep 11 18:23:25.235206 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:25.235169    2578 generic.go:356] "Generic (PLEG): container finished" podID="ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b" containerID="cf8e5eb26023ebf64a38a26815f72753cc6483ffcaa28bb1bf874081f125e51f" exitCode=0
Sep 11 18:23:25.235585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:25.235229    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m" event={"ID":"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b","Type":"ContainerDied","Data":"cf8e5eb26023ebf64a38a26815f72753cc6483ffcaa28bb1bf874081f125e51f"}
Sep 11 18:23:26.241023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:26.240989    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m" event={"ID":"ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b","Type":"ContainerStarted","Data":"90036882753f685ca1d92434fee07a85135c9f385f44dac45dc3c8556108ec97"}
Sep 11 18:23:26.241418 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:26.241114    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:26.265199 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:26.265124    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m" podStartSLOduration=2.831091165 podStartE2EDuration="7.265107828s" totalImagesPullingTime="3.37735491s" totalInitContainerRuntime="1.056661753s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:26.264183706 +0000 UTC m=+469.299400376" watchObservedRunningTime="2026-09-11 18:23:26.265107828 +0000 UTC m=+469.300324498"
Sep 11 18:23:32.222639 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:32.222601    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78b8f7c5cb-zcqqs"
Sep 11 18:23:37.247160 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:37.247123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68b7b6b756-c9p8m"
Sep 11 18:23:48.188396 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.188363    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"]
Sep 11 18:23:48.192818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.192796    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.196018 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.195986    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-q98fw\""
Sep 11 18:23:48.196132 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.196079    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 18:23:48.233236 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.233212    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"]
Sep 11 18:23:48.350336 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.350307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv77n\" (UniqueName: \"kubernetes.io/projected/88e240e9-074e-483f-a292-0b1dc581efb9-kube-api-access-wv77n\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.350336 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.350340    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88e240e9-074e-483f-a292-0b1dc581efb9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.350574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.350364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.350574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.350436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88e240e9-074e-483f-a292-0b1dc581efb9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.350574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.350501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.350574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.350539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.350574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.350557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88e240e9-074e-483f-a292-0b1dc581efb9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.350792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.350588    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.350792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.350627    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.451649 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.451557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.451649 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.451600    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.451649 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.451643    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv77n\" (UniqueName: \"kubernetes.io/projected/88e240e9-074e-483f-a292-0b1dc581efb9-kube-api-access-wv77n\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.451969 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.451663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88e240e9-074e-483f-a292-0b1dc581efb9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.451969 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.451712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.451969 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.451749    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88e240e9-074e-483f-a292-0b1dc581efb9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.451969 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.451776    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.451969 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.451815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.451969 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.451843    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88e240e9-074e-483f-a292-0b1dc581efb9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.452256 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.452141    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.452256 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.452162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.452361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.452277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.452361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.452305    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.452761 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.452739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88e240e9-074e-483f-a292-0b1dc581efb9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.454016 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.453995    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88e240e9-074e-483f-a292-0b1dc581efb9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.454233 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.454217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88e240e9-074e-483f-a292-0b1dc581efb9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.462029 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.462005    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88e240e9-074e-483f-a292-0b1dc581efb9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.462171 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.462153    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv77n\" (UniqueName: \"kubernetes.io/projected/88e240e9-074e-483f-a292-0b1dc581efb9-kube-api-access-wv77n\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk\" (UID: \"88e240e9-074e-483f-a292-0b1dc581efb9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.502999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.502970    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:48.783780 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:48.783753    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"]
Sep 11 18:23:48.785636 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:23:48.785608    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88e240e9_074e_483f_a292_0b1dc581efb9.slice/crio-7ff082492c59650677ba19abd049f685af374167b09eefd43866b68a43cefe76 WatchSource:0}: Error finding container 7ff082492c59650677ba19abd049f685af374167b09eefd43866b68a43cefe76: Status 404 returned error can't find the container with id 7ff082492c59650677ba19abd049f685af374167b09eefd43866b68a43cefe76
Sep 11 18:23:49.345756 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:49.345720    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk" event={"ID":"88e240e9-074e-483f-a292-0b1dc581efb9","Type":"ContainerStarted","Data":"7ff082492c59650677ba19abd049f685af374167b09eefd43866b68a43cefe76"}
Sep 11 18:23:50.682720 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.682668    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g8gpq"]
Sep 11 18:23:50.689191 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.689168    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:50.692427 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.692394    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 18:23:50.693092 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.693066    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-8pxqp\""
Sep 11 18:23:50.700061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.700003    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g8gpq"]
Sep 11 18:23:50.767713 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.767659    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4v5j\" (UniqueName: \"kubernetes.io/projected/04d7afdf-f4a3-42b8-87e5-0612ac2c48b9-kube-api-access-d4v5j\") pod \"odh-model-controller-7998c67dc4-g8gpq\" (UID: \"04d7afdf-f4a3-42b8-87e5-0612ac2c48b9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:50.767900 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.767773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04d7afdf-f4a3-42b8-87e5-0612ac2c48b9-cert\") pod \"odh-model-controller-7998c67dc4-g8gpq\" (UID: \"04d7afdf-f4a3-42b8-87e5-0612ac2c48b9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:50.868643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.868599    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04d7afdf-f4a3-42b8-87e5-0612ac2c48b9-cert\") pod \"odh-model-controller-7998c67dc4-g8gpq\" (UID: \"04d7afdf-f4a3-42b8-87e5-0612ac2c48b9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:50.868827 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.868655    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4v5j\" (UniqueName: \"kubernetes.io/projected/04d7afdf-f4a3-42b8-87e5-0612ac2c48b9-kube-api-access-d4v5j\") pod \"odh-model-controller-7998c67dc4-g8gpq\" (UID: \"04d7afdf-f4a3-42b8-87e5-0612ac2c48b9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:50.868827 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:23:50.868789    2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 11 18:23:50.868919 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:23:50.868872    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d7afdf-f4a3-42b8-87e5-0612ac2c48b9-cert podName:04d7afdf-f4a3-42b8-87e5-0612ac2c48b9 nodeName:}" failed. No retries permitted until 2026-09-11 18:23:51.368849678 +0000 UTC m=+494.404066333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04d7afdf-f4a3-42b8-87e5-0612ac2c48b9-cert") pod "odh-model-controller-7998c67dc4-g8gpq" (UID: "04d7afdf-f4a3-42b8-87e5-0612ac2c48b9") : secret "odh-model-controller-webhook-cert" not found
Sep 11 18:23:50.878283 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:50.878254    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4v5j\" (UniqueName: \"kubernetes.io/projected/04d7afdf-f4a3-42b8-87e5-0612ac2c48b9-kube-api-access-d4v5j\") pod \"odh-model-controller-7998c67dc4-g8gpq\" (UID: \"04d7afdf-f4a3-42b8-87e5-0612ac2c48b9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:51.373753 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:51.373687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04d7afdf-f4a3-42b8-87e5-0612ac2c48b9-cert\") pod \"odh-model-controller-7998c67dc4-g8gpq\" (UID: \"04d7afdf-f4a3-42b8-87e5-0612ac2c48b9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:51.380233 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:51.380199    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04d7afdf-f4a3-42b8-87e5-0612ac2c48b9-cert\") pod \"odh-model-controller-7998c67dc4-g8gpq\" (UID: \"04d7afdf-f4a3-42b8-87e5-0612ac2c48b9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:51.602122 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:51.602086    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:51.647193 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:51.640809    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:23:51.647193 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:51.640904    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:23:51.647193 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:51.640947    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:23:51.905398 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:51.905319    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g8gpq"]
Sep 11 18:23:51.908545 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:23:51.908515    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d7afdf_f4a3_42b8_87e5_0612ac2c48b9.slice/crio-d8ee368e8f9ab5dca657682cf091f153fc4029f896daf2b57d0e738254ee463b WatchSource:0}: Error finding container d8ee368e8f9ab5dca657682cf091f153fc4029f896daf2b57d0e738254ee463b: Status 404 returned error can't find the container with id d8ee368e8f9ab5dca657682cf091f153fc4029f896daf2b57d0e738254ee463b
Sep 11 18:23:52.363651 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:52.363609    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk" event={"ID":"88e240e9-074e-483f-a292-0b1dc581efb9","Type":"ContainerStarted","Data":"2a383c5b62b09d0eee292bc48cdd86307c456eb052bcdd833de0008c19f5a47d"}
Sep 11 18:23:52.364869 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:52.364843    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq" event={"ID":"04d7afdf-f4a3-42b8-87e5-0612ac2c48b9","Type":"ContainerStarted","Data":"d8ee368e8f9ab5dca657682cf091f153fc4029f896daf2b57d0e738254ee463b"}
Sep 11 18:23:52.400428 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:52.400366    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk" podStartSLOduration=1.547433652 podStartE2EDuration="4.40034629s" totalImagesPullingTime="2.852912638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:52.39928232 +0000 UTC m=+495.434498991" watchObservedRunningTime="2026-09-11 18:23:52.40034629 +0000 UTC m=+495.435562962"
Sep 11 18:23:52.503733 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:52.503639    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:52.509177 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:52.509150    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:53.370911 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:53.370873    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:53.372139 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:53.372104    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk"
Sep 11 18:23:55.381022 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:55.380991    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq" event={"ID":"04d7afdf-f4a3-42b8-87e5-0612ac2c48b9","Type":"ContainerStarted","Data":"ac20f279573dd80e21707851f79bdd7cfbb21bbc09c0a7567a1a95d884e75c0d"}
Sep 11 18:23:55.381437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:55.381084    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:23:55.401923 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:55.401879    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq" podStartSLOduration=2.777033977 podStartE2EDuration="5.401865081s" totalImagesPullingTime="2.624831104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:55.400653087 +0000 UTC m=+498.435869767" watchObservedRunningTime="2026-09-11 18:23:55.401865081 +0000 UTC m=+498.437081751"
Sep 11 18:23:58.195499 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.195463    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"]
Sep 11 18:23:58.202609 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.202586    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.205212 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.205191    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 18:23:58.205720 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.205699    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 18:23:58.206023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.206008    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jdl7c\""
Sep 11 18:23:58.206063 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.206030    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 18:23:58.225075 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.225051    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"]
Sep 11 18:23:58.235313 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.235288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.235433 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.235330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.235433 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.235373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.235433 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.235399    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kqsm\" (UniqueName: \"kubernetes.io/projected/b911bd70-ea94-468f-92b2-4bcf6b156d31-kube-api-access-6kqsm\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.235606 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.235443    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.235606 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.235492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b911bd70-ea94-468f-92b2-4bcf6b156d31-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.235606 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.235523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b911bd70-ea94-468f-92b2-4bcf6b156d31-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.335908 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.335870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.335908 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.335914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.336125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.335941    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kqsm\" (UniqueName: \"kubernetes.io/projected/b911bd70-ea94-468f-92b2-4bcf6b156d31-kube-api-access-6kqsm\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.336125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.335970    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.336125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.336008    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b911bd70-ea94-468f-92b2-4bcf6b156d31-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.336125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.336053    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b911bd70-ea94-468f-92b2-4bcf6b156d31-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.336125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.336086    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.336728 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.336697    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.338342 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.338312    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b911bd70-ea94-468f-92b2-4bcf6b156d31-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.338557 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.338537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.338648 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.338626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.338729 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.338625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b911bd70-ea94-468f-92b2-4bcf6b156d31-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.368621 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.368590    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kqsm\" (UniqueName: \"kubernetes.io/projected/b911bd70-ea94-468f-92b2-4bcf6b156d31-kube-api-access-6kqsm\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.374040 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.374017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b911bd70-ea94-468f-92b2-4bcf6b156d31-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-vpmf6\" (UID: \"b911bd70-ea94-468f-92b2-4bcf6b156d31\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.511332 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.511239    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:23:58.795641 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:58.795610    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"]
Sep 11 18:23:58.796359 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:23:58.796334    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb911bd70_ea94_468f_92b2_4bcf6b156d31.slice/crio-4d79843834a703f383945d033242bc3b7db5e4ceeb505543968ab8ea4af3af76 WatchSource:0}: Error finding container 4d79843834a703f383945d033242bc3b7db5e4ceeb505543968ab8ea4af3af76: Status 404 returned error can't find the container with id 4d79843834a703f383945d033242bc3b7db5e4ceeb505543968ab8ea4af3af76
Sep 11 18:23:59.400030 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:23:59.399995    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6" event={"ID":"b911bd70-ea94-468f-92b2-4bcf6b156d31","Type":"ContainerStarted","Data":"4d79843834a703f383945d033242bc3b7db5e4ceeb505543968ab8ea4af3af76"}
Sep 11 18:24:01.438896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:24:01.438858    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:24:01.439287 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:24:01.438936    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:24:02.417373 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:24:02.417332    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6" event={"ID":"b911bd70-ea94-468f-92b2-4bcf6b156d31","Type":"ContainerStarted","Data":"65a2567ac257916913ebdf00a69527a87d7529f80ce0a7fd11c6293e7daa8f80"}
Sep 11 18:24:02.417570 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:24:02.417494    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:24:02.419328 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:24:02.419303    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-vpmf6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:24:02.419441 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:24:02.419347    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6" podUID="b911bd70-ea94-468f-92b2-4bcf6b156d31" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:24:02.452181 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:24:02.452130    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6" podStartSLOduration=1.811906527 podStartE2EDuration="4.452111812s" totalImagesPullingTime="2.640205285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:24:02.450268558 +0000 UTC m=+505.485485228" watchObservedRunningTime="2026-09-11 18:24:02.452111812 +0000 UTC m=+505.487328483"
Sep 11 18:24:03.422982 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:24:03.422951    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-vpmf6"
Sep 11 18:24:06.388213 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:24:06.388184    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-g8gpq"
Sep 11 18:25:37.380622 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:25:37.380591    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:25:37.381237 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:25:37.380806    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:25:37.385197 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:25:37.385176    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:25:37.385446 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:25:37.385420    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:26:21.700728 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.700695    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"]
Sep 11 18:26:21.703864 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.703842    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.707817 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.707792    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ggwqw\""
Sep 11 18:26:21.719426 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.719405    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"]
Sep 11 18:26:21.826723 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.826665    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2m9m\" (UniqueName: \"kubernetes.io/projected/e701e4f0-fa92-4cd0-ae56-737249752574-kube-api-access-q2m9m\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.826887 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.826747    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e701e4f0-fa92-4cd0-ae56-737249752574-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.826887 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.826775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.826887 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.826812    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.826887 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.826853    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.826887 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.826877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.827133 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.826908    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.827133 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.826993    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e701e4f0-fa92-4cd0-ae56-737249752574-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.827133 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.827031    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e701e4f0-fa92-4cd0-ae56-737249752574-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.928434 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928395    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e701e4f0-fa92-4cd0-ae56-737249752574-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.928615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928440    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.928615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.928615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.928615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928538    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.928615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.928615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e701e4f0-fa92-4cd0-ae56-737249752574-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.928952 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e701e4f0-fa92-4cd0-ae56-737249752574-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.928952 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928724    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2m9m\" (UniqueName: \"kubernetes.io/projected/e701e4f0-fa92-4cd0-ae56-737249752574-kube-api-access-q2m9m\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.929045 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.928996    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.929262 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.929205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.929262 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.929239    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.929458 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.929269    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e701e4f0-fa92-4cd0-ae56-737249752574-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.929458 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.929397    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.930890 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.930866    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e701e4f0-fa92-4cd0-ae56-737249752574-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.931445 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.931424    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e701e4f0-fa92-4cd0-ae56-737249752574-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.938126 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.938094    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e701e4f0-fa92-4cd0-ae56-737249752574-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:21.938323 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:21.938302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2m9m\" (UniqueName: \"kubernetes.io/projected/e701e4f0-fa92-4cd0-ae56-737249752574-kube-api-access-q2m9m\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dwbw9\" (UID: \"e701e4f0-fa92-4cd0-ae56-737249752574\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:22.013635 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:22.013555    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:22.302649 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:22.302626    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"]
Sep 11 18:26:22.304393 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:26:22.304369    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode701e4f0_fa92_4cd0_ae56_737249752574.slice/crio-44b44009786925a8d9ce13372609998ba7285858df0d4d38cd5e0fccc355572a WatchSource:0}: Error finding container 44b44009786925a8d9ce13372609998ba7285858df0d4d38cd5e0fccc355572a: Status 404 returned error can't find the container with id 44b44009786925a8d9ce13372609998ba7285858df0d4d38cd5e0fccc355572a
Sep 11 18:26:22.306557 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:22.306520    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:26:22.306634 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:22.306588    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:26:22.306634 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:22.306619    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:26:23.031871 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:23.031832    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" event={"ID":"e701e4f0-fa92-4cd0-ae56-737249752574","Type":"ContainerStarted","Data":"c7c38e2635cbba5960ac1691a6ab966f32d5e8fee874e1c204934ec21335a31c"}
Sep 11 18:26:23.032364 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:23.031930    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" event={"ID":"e701e4f0-fa92-4cd0-ae56-737249752574","Type":"ContainerStarted","Data":"44b44009786925a8d9ce13372609998ba7285858df0d4d38cd5e0fccc355572a"}
Sep 11 18:26:23.056625 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:23.056573    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" podStartSLOduration=2.056555673 podStartE2EDuration="2.056555673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:26:23.054257649 +0000 UTC m=+646.089474318" watchObservedRunningTime="2026-09-11 18:26:23.056555673 +0000 UTC m=+646.091772348"
Sep 11 18:26:24.014762 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:24.014720    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:24.019829 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:24.019805    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:24.036709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:24.036669    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:26:24.037531 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:26:24.037513    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:27:52.548119 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.548043    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"]
Sep 11 18:27:52.551272 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.551252    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:27:52.554832 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.554809    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 18:27:52.555431 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.555415    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-knj28\""
Sep 11 18:27:52.566776 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.566755    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"]
Sep 11 18:27:52.674278 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.674242    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krd2r\" (UniqueName: \"kubernetes.io/projected/7ddd784b-815b-42c0-bf68-667587dda11b-kube-api-access-krd2r\") pod \"payload-processing-5b54b7fbb7-lnhlh\" (UID: \"7ddd784b-815b-42c0-bf68-667587dda11b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:27:52.674430 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.674365    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ddd784b-815b-42c0-bf68-667587dda11b-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-lnhlh\" (UID: \"7ddd784b-815b-42c0-bf68-667587dda11b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:27:52.775459 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.775418    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ddd784b-815b-42c0-bf68-667587dda11b-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-lnhlh\" (UID: \"7ddd784b-815b-42c0-bf68-667587dda11b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:27:52.775660 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.775490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krd2r\" (UniqueName: \"kubernetes.io/projected/7ddd784b-815b-42c0-bf68-667587dda11b-kube-api-access-krd2r\") pod \"payload-processing-5b54b7fbb7-lnhlh\" (UID: \"7ddd784b-815b-42c0-bf68-667587dda11b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:27:52.776234 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.776207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ddd784b-815b-42c0-bf68-667587dda11b-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-lnhlh\" (UID: \"7ddd784b-815b-42c0-bf68-667587dda11b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:27:52.785753 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.785724    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krd2r\" (UniqueName: \"kubernetes.io/projected/7ddd784b-815b-42c0-bf68-667587dda11b-kube-api-access-krd2r\") pod \"payload-processing-5b54b7fbb7-lnhlh\" (UID: \"7ddd784b-815b-42c0-bf68-667587dda11b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:27:52.861561 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:52.861491    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:27:53.157820 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:53.157792    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"]
Sep 11 18:27:53.158997 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:27:53.158958    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ddd784b_815b_42c0_bf68_667587dda11b.slice/crio-80e520fcca3e1211f53c6f511b8a868f724d11c0f0cbea0102edf25358632a5a WatchSource:0}: Error finding container 80e520fcca3e1211f53c6f511b8a868f724d11c0f0cbea0102edf25358632a5a: Status 404 returned error can't find the container with id 80e520fcca3e1211f53c6f511b8a868f724d11c0f0cbea0102edf25358632a5a
Sep 11 18:27:53.161052 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:53.161032    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:27:53.433788 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:53.433741    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh" event={"ID":"7ddd784b-815b-42c0-bf68-667587dda11b","Type":"ContainerStarted","Data":"80e520fcca3e1211f53c6f511b8a868f724d11c0f0cbea0102edf25358632a5a"}
Sep 11 18:27:56.450067 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:56.450029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh" event={"ID":"7ddd784b-815b-42c0-bf68-667587dda11b","Type":"ContainerStarted","Data":"1ae6dd0e4c44e93b98d1508b74b4a3ae4c576694b9730269fbfb579f2aaae364"}
Sep 11 18:27:56.450456 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:56.450104    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:27:56.470650 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:27:56.470605    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh" podStartSLOduration=1.744810836 podStartE2EDuration="4.47058953s" totalImagesPullingTime="2.725778694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:27:56.468398897 +0000 UTC m=+739.503615582" watchObservedRunningTime="2026-09-11 18:27:56.47058953 +0000 UTC m=+739.505806199"
Sep 11 18:28:02.480187 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:02.480154    2578 generic.go:356] "Generic (PLEG): container finished" podID="e701e4f0-fa92-4cd0-ae56-737249752574" containerID="c7c38e2635cbba5960ac1691a6ab966f32d5e8fee874e1c204934ec21335a31c" exitCode=0
Sep 11 18:28:02.480187 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:02.480190    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" event={"ID":"e701e4f0-fa92-4cd0-ae56-737249752574","Type":"ContainerDied","Data":"c7c38e2635cbba5960ac1691a6ab966f32d5e8fee874e1c204934ec21335a31c"}
Sep 11 18:28:02.480666 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:02.480574    2578 scope.go:117] "RemoveContainer" containerID="c7c38e2635cbba5960ac1691a6ab966f32d5e8fee874e1c204934ec21335a31c"
Sep 11 18:28:02.481554 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:02.481518    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:28:02.481658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:02.481587    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:28:02.481658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:02.481619    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:28:03.486707 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:03.486659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" event={"ID":"e701e4f0-fa92-4cd0-ae56-737249752574","Type":"ContainerStarted","Data":"ee21d80dc75475fe7001ea28b5441fddd67a4ff559e79313caa524c511d99eeb"}
Sep 11 18:28:04.013963 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:04.013924    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:28:04.014213 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:04.014183    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dwbw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 11 18:28:04.014285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:04.014255    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" podUID="e701e4f0-fa92-4cd0-ae56-737249752574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 11 18:28:05.014019 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:05.013982    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dwbw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 11 18:28:05.015008 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:05.014564    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" podUID="e701e4f0-fa92-4cd0-ae56-737249752574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 11 18:28:06.014339 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:06.014296    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dwbw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 11 18:28:06.014854 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:06.014374    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" podUID="e701e4f0-fa92-4cd0-ae56-737249752574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 11 18:28:07.014829 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:07.014787    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dwbw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 11 18:28:07.015418 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:07.015388    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" podUID="e701e4f0-fa92-4cd0-ae56-737249752574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 11 18:28:07.455799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:07.455768    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-lnhlh"
Sep 11 18:28:08.014810 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:08.014774    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dwbw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 11 18:28:08.015084 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:08.015054    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9" podUID="e701e4f0-fa92-4cd0-ae56-737249752574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 11 18:28:09.020782 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:09.020750    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:28:09.021970 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:09.021942    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:28:09.022658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:28:09.022619    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dwbw9"
Sep 11 18:29:36.648004 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.647970    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"]
Sep 11 18:29:36.650305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.650289    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.653634 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.653609    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 18:29:36.653792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.653646    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 18:29:36.653792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.653719    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 11 18:29:36.653792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.653729    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-x8pc2\""
Sep 11 18:29:36.660309 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.660288    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"]
Sep 11 18:29:36.808202 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.808165    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.808361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.808205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.808361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.808292    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.808361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.808321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.808361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.808351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scb42\" (UniqueName: \"kubernetes.io/projected/7d4bccbe-b752-4345-9e63-7db534a07a69-kube-api-access-scb42\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.808490 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.808417    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d4bccbe-b752-4345-9e63-7db534a07a69-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.909615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.909588    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.909815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.909621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.909815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.909645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scb42\" (UniqueName: \"kubernetes.io/projected/7d4bccbe-b752-4345-9e63-7db534a07a69-kube-api-access-scb42\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.909815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.909663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d4bccbe-b752-4345-9e63-7db534a07a69-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.909815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.909736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.909815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.909772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.910050 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.910027    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.910100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.910065    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.910136 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.910118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.911992 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.911970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d4bccbe-b752-4345-9e63-7db534a07a69-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.912264 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.912244    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d4bccbe-b752-4345-9e63-7db534a07a69-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.917479 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.917457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scb42\" (UniqueName: \"kubernetes.io/projected/7d4bccbe-b752-4345-9e63-7db534a07a69-kube-api-access-scb42\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dmqhg\" (UID: \"7d4bccbe-b752-4345-9e63-7db534a07a69\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:36.961471 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:36.961449    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:37.232001 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:37.231975    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"]
Sep 11 18:29:37.233818 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:29:37.233793    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4bccbe_b752_4345_9e63_7db534a07a69.slice/crio-97bf2aee413edd692b499ecc252f6810bbfb73d9043ea40613b2069d68050eb4 WatchSource:0}: Error finding container 97bf2aee413edd692b499ecc252f6810bbfb73d9043ea40613b2069d68050eb4: Status 404 returned error can't find the container with id 97bf2aee413edd692b499ecc252f6810bbfb73d9043ea40613b2069d68050eb4
Sep 11 18:29:37.913241 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:37.913203    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg" event={"ID":"7d4bccbe-b752-4345-9e63-7db534a07a69","Type":"ContainerStarted","Data":"97bf2aee413edd692b499ecc252f6810bbfb73d9043ea40613b2069d68050eb4"}
Sep 11 18:29:38.920227 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:38.920190    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg" event={"ID":"7d4bccbe-b752-4345-9e63-7db534a07a69","Type":"ContainerStarted","Data":"127fa10519962fc417bc464c37d57900752e1276d352871e74e244eccbb579d6"}
Sep 11 18:29:38.937480 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:38.937423    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg" podStartSLOduration=1.9305532090000002 podStartE2EDuration="2.937408664s" totalImagesPullingTime="1.006855455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:29:38.936411646 +0000 UTC m=+841.971628313" watchObservedRunningTime="2026-09-11 18:29:38.937408664 +0000 UTC m=+841.972625332"
Sep 11 18:29:46.961986 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:46.961954    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:46.962542 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:46.962066    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:46.974288 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:46.974256    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:29:47.970930 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:29:47.970904    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dmqhg"
Sep 11 18:30:00.145657 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:00.145618    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn"]
Sep 11 18:30:00.148044 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:00.148026    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn"
Sep 11 18:30:00.150577 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:00.150560    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:30:00.157083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:00.157061    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn"]
Sep 11 18:30:00.158524 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:00.158506    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn"
Sep 11 18:30:00.426970 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:00.426897    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn"]
Sep 11 18:30:00.429051 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:30:00.429024    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e613513_91d3_4248_81a4_fc6b038b1339.slice/crio-2d0c1896b3ac644e9b60292b10fe0b99e5cf30ea5b72969caa041e33b5fe32a3 WatchSource:0}: Error finding container 2d0c1896b3ac644e9b60292b10fe0b99e5cf30ea5b72969caa041e33b5fe32a3: Status 404 returned error can't find the container with id 2d0c1896b3ac644e9b60292b10fe0b99e5cf30ea5b72969caa041e33b5fe32a3
Sep 11 18:30:01.020131 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:01.020094    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn" event={"ID":"1e613513-91d3-4248-81a4-fc6b038b1339","Type":"ContainerStarted","Data":"2d0c1896b3ac644e9b60292b10fe0b99e5cf30ea5b72969caa041e33b5fe32a3"}
Sep 11 18:30:03.030807 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:03.030713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn" event={"ID":"1e613513-91d3-4248-81a4-fc6b038b1339","Type":"ContainerStarted","Data":"98f91cf25e7b93f32db0f33cd82dc254c37968db9a7149b27f3e576d301575c0"}
Sep 11 18:30:03.046964 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:03.046911    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn" podStartSLOduration=1.07164291 podStartE2EDuration="3.046895454s" totalImagesPullingTime="1.975252544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:30:03.045947036 +0000 UTC m=+866.081163706" watchObservedRunningTime="2026-09-11 18:30:03.046895454 +0000 UTC m=+866.082112122"
Sep 11 18:30:04.036390 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:04.036356    2578 generic.go:356] "Generic (PLEG): container finished" podID="1e613513-91d3-4248-81a4-fc6b038b1339" containerID="98f91cf25e7b93f32db0f33cd82dc254c37968db9a7149b27f3e576d301575c0" exitCode=0
Sep 11 18:30:04.036856 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:04.036450    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn" event={"ID":"1e613513-91d3-4248-81a4-fc6b038b1339","Type":"ContainerDied","Data":"98f91cf25e7b93f32db0f33cd82dc254c37968db9a7149b27f3e576d301575c0"}
Sep 11 18:30:05.163495 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:05.163466    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn"
Sep 11 18:30:06.046596 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:06.046564    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn"
Sep 11 18:30:06.046839 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:06.046566    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819190-s5wpn" event={"ID":"1e613513-91d3-4248-81a4-fc6b038b1339","Type":"ContainerDied","Data":"2d0c1896b3ac644e9b60292b10fe0b99e5cf30ea5b72969caa041e33b5fe32a3"}
Sep 11 18:30:06.046839 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:06.046730    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d0c1896b3ac644e9b60292b10fe0b99e5cf30ea5b72969caa041e33b5fe32a3"
Sep 11 18:30:37.414156 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:37.414082    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:30:37.417833 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:37.417807    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:30:37.418561 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:37.418539    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:30:37.421776 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:30:37.421758    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:31:58.601201 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.601163    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"]
Sep 11 18:31:58.601722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.601562    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e613513-91d3-4248-81a4-fc6b038b1339" containerName="cleanup"
Sep 11 18:31:58.601722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.601578    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e613513-91d3-4248-81a4-fc6b038b1339" containerName="cleanup"
Sep 11 18:31:58.601722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.601650    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e613513-91d3-4248-81a4-fc6b038b1339" containerName="cleanup"
Sep 11 18:31:58.604512 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.604497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.608163 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.608139    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-047453-openshift-default-dockercfg-krjlm\""
Sep 11 18:31:58.626353 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.626323    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"]
Sep 11 18:31:58.686304 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.686265    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"]
Sep 11 18:31:58.689706 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.689664    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.693105 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.693075    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-200c07-openshift-default-dockercfg-82m8t\""
Sep 11 18:31:58.704815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.704789    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"]
Sep 11 18:31:58.706945 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.706918    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-data\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.707059 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.706976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-socket\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.707059 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.707005    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-podinfo\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.707059 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.707037    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-credential-socket\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.707219 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.707077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-certs\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.707219 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.707111    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgd6x\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-kube-api-access-bgd6x\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.707219 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.707140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-envoy\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.707219 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.707201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istiod-ca-cert\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.707353 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.707236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-token\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.760256 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.760225    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"]
Sep 11 18:31:58.764475 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.764450    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.767406 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.767380    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4d5a75-openshift-default-dockercfg-j8nc2\""
Sep 11 18:31:58.778347 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.778318    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"]
Sep 11 18:31:58.807896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.807859    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-token\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808069 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.807906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-credential-socket\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.808069 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.807935    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-envoy\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.808069 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.807966    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-socket\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.808069 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.807995    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-data\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808069 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808014    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-data\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.808371 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-socket\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808371 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-podinfo\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808371 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808246    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-podinfo\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.808371 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-credential-socket\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808414    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm6w6\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-kube-api-access-tm6w6\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.808540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808457    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-envoy\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.808540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-data\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808485    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-data\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.808540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-socket\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfx7d\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-kube-api-access-vfx7d\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.808837 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istiod-ca-cert\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.808837 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808572    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-credential-socket\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808837 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-socket\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.808837 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-certs\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.808837 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-certs\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808837 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-token\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.808837 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808754    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgd6x\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-kube-api-access-bgd6x\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.808837 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808804    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-token\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.808837 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808835    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-envoy\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.809368 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-podinfo\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.809368 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808887    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istiod-ca-cert\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.809368 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istiod-ca-cert\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.809368 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808946    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-credential-socket\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.809368 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.808980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-certs\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.809612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.809371    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-certs\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.809740 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.809720    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istiod-ca-cert\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.811346 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.811322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-envoy\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.811436 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.811325    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-podinfo\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.819986 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.819956    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgd6x\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-kube-api-access-bgd6x\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.820241 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.820221    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-token\") pod \"e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") " pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.910293 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-socket\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.910460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-certs\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.910460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910318    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-token\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.910460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-token\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.910460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910382    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-podinfo\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.910460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istiod-ca-cert\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.910460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-credential-socket\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-certs\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910502    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-credential-socket\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910524    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-envoy\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-socket\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-data\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-podinfo\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm6w6\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-kube-api-access-tm6w6\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910786    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-envoy\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910793    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-socket\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910807    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-data\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.910842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910832    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfx7d\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-kube-api-access-vfx7d\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.911449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istiod-ca-cert\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.911449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910898    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-certs\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.911449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.910983    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-certs\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.911449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.911287    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istiod-ca-cert\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.911449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.911388    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-data\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.911727 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.911636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-credential-socket\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.911960 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.911297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-credential-socket\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.913608 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.913580    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-socket\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.913867 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.913841    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-data\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.913953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.913909    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-envoy\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.914109 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.914088    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:31:58.914311 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.914288    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-podinfo\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.914586 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.914567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-podinfo\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.914742 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.914700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istiod-ca-cert\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.915963 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.915942    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-envoy\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.920178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.920152    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-token\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:58.920879 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.920854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-token\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.921017 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.920996    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfx7d\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-kube-api-access-vfx7d\") pod \"e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") " pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:58.921257 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.921238    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm6w6\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-kube-api-access-tm6w6\") pod \"e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") " pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:59.000524 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:58.999935    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:31:59.076532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.076497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:31:59.196120 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.196094    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"]
Sep 11 18:31:59.198122 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:31:59.198092    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f1cbe6_60fc_4c20_8b53_c4b9223427ee.slice/crio-959380fdb217017a55be6ba927a254858481db83d456f88e7fe738e2c60d817c WatchSource:0}: Error finding container 959380fdb217017a55be6ba927a254858481db83d456f88e7fe738e2c60d817c: Status 404 returned error can't find the container with id 959380fdb217017a55be6ba927a254858481db83d456f88e7fe738e2c60d817c
Sep 11 18:31:59.200261 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.200230    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:31:59.200337 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.200297    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:31:59.200337 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.200325    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:31:59.299267 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.299204    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"]
Sep 11 18:31:59.301957 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:31:59.301926    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d90e55_a5ce_4d17_9c2f_8b46344193c5.slice/crio-085c5ad5dd17c907bd15bee582c14d29e369329f1851f621be72e0eb721922c6 WatchSource:0}: Error finding container 085c5ad5dd17c907bd15bee582c14d29e369329f1851f621be72e0eb721922c6: Status 404 returned error can't find the container with id 085c5ad5dd17c907bd15bee582c14d29e369329f1851f621be72e0eb721922c6
Sep 11 18:31:59.304266 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.304233    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:31:59.304344 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.304317    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:31:59.304381 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.304358    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:31:59.357206 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.357179    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"]
Sep 11 18:31:59.359700 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:31:59.359647    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6fc16f_db5c_4df4_8d13_5911feba2ee2.slice/crio-826310f913a80f40f108c1b4846a7c76b805039b2f181d7bb0d6653fa05a4098 WatchSource:0}: Error finding container 826310f913a80f40f108c1b4846a7c76b805039b2f181d7bb0d6653fa05a4098: Status 404 returned error can't find the container with id 826310f913a80f40f108c1b4846a7c76b805039b2f181d7bb0d6653fa05a4098
Sep 11 18:31:59.362574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.362541    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:31:59.362760 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.362609    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:31:59.362760 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.362647    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:31:59.574583 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.574482    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" event={"ID":"bd6fc16f-db5c-4df4-8d13-5911feba2ee2","Type":"ContainerStarted","Data":"1acf7b775eb9fec9290d2d22b6404da69510494684f8e7ddd7af40cd932e2815"}
Sep 11 18:31:59.574583 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.574542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" event={"ID":"bd6fc16f-db5c-4df4-8d13-5911feba2ee2","Type":"ContainerStarted","Data":"826310f913a80f40f108c1b4846a7c76b805039b2f181d7bb0d6653fa05a4098"}
Sep 11 18:31:59.576591 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.576545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" event={"ID":"44d90e55-a5ce-4d17-9c2f-8b46344193c5","Type":"ContainerStarted","Data":"45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1"}
Sep 11 18:31:59.576766 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.576601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" event={"ID":"44d90e55-a5ce-4d17-9c2f-8b46344193c5","Type":"ContainerStarted","Data":"085c5ad5dd17c907bd15bee582c14d29e369329f1851f621be72e0eb721922c6"}
Sep 11 18:31:59.578848 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.578822    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" event={"ID":"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee","Type":"ContainerStarted","Data":"83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7"}
Sep 11 18:31:59.578974 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.578867    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" event={"ID":"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee","Type":"ContainerStarted","Data":"959380fdb217017a55be6ba927a254858481db83d456f88e7fe738e2c60d817c"}
Sep 11 18:31:59.594484 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.594422    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podStartSLOduration=1.5944012829999998 podStartE2EDuration="1.594401283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:31:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:31:59.593727788 +0000 UTC m=+982.628944455" watchObservedRunningTime="2026-09-11 18:31:59.594401283 +0000 UTC m=+982.629617954"
Sep 11 18:31:59.613451 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.613395    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" podStartSLOduration=1.6133792850000002 podStartE2EDuration="1.613379285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:31:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:31:59.611120693 +0000 UTC m=+982.646337365" watchObservedRunningTime="2026-09-11 18:31:59.613379285 +0000 UTC m=+982.648595955"
Sep 11 18:31:59.633839 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.633774    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podStartSLOduration=1.6337524829999999 podStartE2EDuration="1.633752483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:31:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:31:59.633397885 +0000 UTC m=+982.668614558" watchObservedRunningTime="2026-09-11 18:31:59.633752483 +0000 UTC m=+982.668969153"
Sep 11 18:31:59.915385 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:31:59.915353    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:32:00.001085 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:00.001041    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:32:00.006818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:00.006787    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:32:00.076877 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:00.076832    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:32:00.078205 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:00.078181    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 18:32:00.078290 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:00.078233    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 18:32:00.590600 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:00.590565    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:32:00.591249 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:00.591228    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:32:00.920387 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:00.920359    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:32:01.081306 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:01.081276    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:32:01.595177 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:01.595142    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:32:01.595382 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:01.595303    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:32:01.596093 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:01.596071    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:32:01.596242 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:01.596224    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:32:04.599015 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.598969    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"]
Sep 11 18:32:04.601221 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.601203    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:04.603861 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.603837    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-200c07\""
Sep 11 18:32:04.603994 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.603978    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-200c07-dockercfg-qvcmm\""
Sep 11 18:32:04.612112 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.612083    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"]
Sep 11 18:32:04.672568 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.672532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/644c1a84-c849-4a1f-8ba5-3c358d956678-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql\" (UID: \"644c1a84-c849-4a1f-8ba5-3c358d956678\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:04.672751 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.672576    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcd9z\" (UniqueName: \"kubernetes.io/projected/644c1a84-c849-4a1f-8ba5-3c358d956678-kube-api-access-gcd9z\") pod \"payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql\" (UID: \"644c1a84-c849-4a1f-8ba5-3c358d956678\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:04.774094 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.774051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/644c1a84-c849-4a1f-8ba5-3c358d956678-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql\" (UID: \"644c1a84-c849-4a1f-8ba5-3c358d956678\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:04.774286 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.774099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcd9z\" (UniqueName: \"kubernetes.io/projected/644c1a84-c849-4a1f-8ba5-3c358d956678-kube-api-access-gcd9z\") pod \"payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql\" (UID: \"644c1a84-c849-4a1f-8ba5-3c358d956678\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:04.774876 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.774853    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/644c1a84-c849-4a1f-8ba5-3c358d956678-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql\" (UID: \"644c1a84-c849-4a1f-8ba5-3c358d956678\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:04.782063 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.782035    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcd9z\" (UniqueName: \"kubernetes.io/projected/644c1a84-c849-4a1f-8ba5-3c358d956678-kube-api-access-gcd9z\") pod \"payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql\" (UID: \"644c1a84-c849-4a1f-8ba5-3c358d956678\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:04.913444 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:04.913415    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:05.190764 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.187316    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"]
Sep 11 18:32:05.196780 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:32:05.196740    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644c1a84_c849_4a1f_8ba5_3c358d956678.slice/crio-93a714458b643b42770d139b6ce07d22496be0eaef716f55088305fcf29ec31f WatchSource:0}: Error finding container 93a714458b643b42770d139b6ce07d22496be0eaef716f55088305fcf29ec31f: Status 404 returned error can't find the container with id 93a714458b643b42770d139b6ce07d22496be0eaef716f55088305fcf29ec31f
Sep 11 18:32:05.506624 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.506540    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"]
Sep 11 18:32:05.508991 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.508972    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:05.511890 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.511859    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-047453-dockercfg-kxdnh\""
Sep 11 18:32:05.512002 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.511898    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-047453\""
Sep 11 18:32:05.520610 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.520585    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"]
Sep 11 18:32:05.532029 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.532002    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"]
Sep 11 18:32:05.534748 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.534724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:05.551425 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.551399    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"]
Sep 11 18:32:05.583658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.583610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589e13de-5fc0-44e4-848c-4bf774d2fde6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9\" (UID: \"589e13de-5fc0-44e4-848c-4bf774d2fde6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:05.583865 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.583808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hstrw\" (UniqueName: \"kubernetes.io/projected/589e13de-5fc0-44e4-848c-4bf774d2fde6-kube-api-access-hstrw\") pod \"payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9\" (UID: \"589e13de-5fc0-44e4-848c-4bf774d2fde6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:05.623673 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.620750    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql" event={"ID":"644c1a84-c849-4a1f-8ba5-3c358d956678","Type":"ContainerStarted","Data":"03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa"}
Sep 11 18:32:05.623673 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.620850    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql" event={"ID":"644c1a84-c849-4a1f-8ba5-3c358d956678","Type":"ContainerStarted","Data":"93a714458b643b42770d139b6ce07d22496be0eaef716f55088305fcf29ec31f"}
Sep 11 18:32:05.623673 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.621886    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:05.648754 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.648699    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql" podStartSLOduration=1.6486621320000001 podStartE2EDuration="1.648662132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:05.647012437 +0000 UTC m=+988.682229109" watchObservedRunningTime="2026-09-11 18:32:05.648662132 +0000 UTC m=+988.683878806"
Sep 11 18:32:05.685178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.685134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hstrw\" (UniqueName: \"kubernetes.io/projected/589e13de-5fc0-44e4-848c-4bf774d2fde6-kube-api-access-hstrw\") pod \"payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9\" (UID: \"589e13de-5fc0-44e4-848c-4bf774d2fde6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:05.685178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.685179    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51d68738-4236-4ed2-8367-13bb27412e8f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675\" (UID: \"51d68738-4236-4ed2-8367-13bb27412e8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:05.685456 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.685212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lxkv\" (UniqueName: \"kubernetes.io/projected/51d68738-4236-4ed2-8367-13bb27412e8f-kube-api-access-5lxkv\") pod \"payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675\" (UID: \"51d68738-4236-4ed2-8367-13bb27412e8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:05.685456 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.685282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589e13de-5fc0-44e4-848c-4bf774d2fde6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9\" (UID: \"589e13de-5fc0-44e4-848c-4bf774d2fde6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:05.685993 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.685972    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589e13de-5fc0-44e4-848c-4bf774d2fde6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9\" (UID: \"589e13de-5fc0-44e4-848c-4bf774d2fde6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:05.693324 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.693296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hstrw\" (UniqueName: \"kubernetes.io/projected/589e13de-5fc0-44e4-848c-4bf774d2fde6-kube-api-access-hstrw\") pod \"payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9\" (UID: \"589e13de-5fc0-44e4-848c-4bf774d2fde6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:05.786797 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.786696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51d68738-4236-4ed2-8367-13bb27412e8f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675\" (UID: \"51d68738-4236-4ed2-8367-13bb27412e8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:05.786797 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.786767    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lxkv\" (UniqueName: \"kubernetes.io/projected/51d68738-4236-4ed2-8367-13bb27412e8f-kube-api-access-5lxkv\") pod \"payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675\" (UID: \"51d68738-4236-4ed2-8367-13bb27412e8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:05.787454 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.787431    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51d68738-4236-4ed2-8367-13bb27412e8f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675\" (UID: \"51d68738-4236-4ed2-8367-13bb27412e8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:05.795567 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.795536    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lxkv\" (UniqueName: \"kubernetes.io/projected/51d68738-4236-4ed2-8367-13bb27412e8f-kube-api-access-5lxkv\") pod \"payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675\" (UID: \"51d68738-4236-4ed2-8367-13bb27412e8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:05.822397 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.822348    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:05.846228 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:05.846189    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:06.109767 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:06.109733    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"]
Sep 11 18:32:06.111881 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:32:06.111846    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589e13de_5fc0_44e4_848c_4bf774d2fde6.slice/crio-514cfa4c5ef52f7640ce0821a94dadbb332e8accd29dbf7308a95a74c3d41715 WatchSource:0}: Error finding container 514cfa4c5ef52f7640ce0821a94dadbb332e8accd29dbf7308a95a74c3d41715: Status 404 returned error can't find the container with id 514cfa4c5ef52f7640ce0821a94dadbb332e8accd29dbf7308a95a74c3d41715
Sep 11 18:32:06.147813 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:06.147783    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"]
Sep 11 18:32:06.149508 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:32:06.149483    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d68738_4236_4ed2_8367_13bb27412e8f.slice/crio-b5307384202bba277fdc3388e47237b6640a698cb7de8ac22a18b8c5ea00f6ae WatchSource:0}: Error finding container b5307384202bba277fdc3388e47237b6640a698cb7de8ac22a18b8c5ea00f6ae: Status 404 returned error can't find the container with id b5307384202bba277fdc3388e47237b6640a698cb7de8ac22a18b8c5ea00f6ae
Sep 11 18:32:06.629184 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:06.629141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9" event={"ID":"589e13de-5fc0-44e4-848c-4bf774d2fde6","Type":"ContainerStarted","Data":"36b1764aee17a62a0038043b09a60d80aa9417e66a15de49c7a3e87699c797c0"}
Sep 11 18:32:06.629830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:06.629751    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9" event={"ID":"589e13de-5fc0-44e4-848c-4bf774d2fde6","Type":"ContainerStarted","Data":"514cfa4c5ef52f7640ce0821a94dadbb332e8accd29dbf7308a95a74c3d41715"}
Sep 11 18:32:06.629830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:06.629795    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:06.631331 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:06.631304    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675" event={"ID":"51d68738-4236-4ed2-8367-13bb27412e8f","Type":"ContainerStarted","Data":"d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a"}
Sep 11 18:32:06.631458 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:06.631362    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675" event={"ID":"51d68738-4236-4ed2-8367-13bb27412e8f","Type":"ContainerStarted","Data":"b5307384202bba277fdc3388e47237b6640a698cb7de8ac22a18b8c5ea00f6ae"}
Sep 11 18:32:06.646501 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:06.646423    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9" podStartSLOduration=1.6464041520000001 podStartE2EDuration="1.646404152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:06.64617834 +0000 UTC m=+989.681395023" watchObservedRunningTime="2026-09-11 18:32:06.646404152 +0000 UTC m=+989.681620822"
Sep 11 18:32:06.663394 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:06.663342    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675" podStartSLOduration=1.6633269529999999 podStartE2EDuration="1.663326953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:06.661316818 +0000 UTC m=+989.696533488" watchObservedRunningTime="2026-09-11 18:32:06.663326953 +0000 UTC m=+989.698543623"
Sep 11 18:32:07.637113 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:07.637071    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:07.908641 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:07.908605    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"]
Sep 11 18:32:07.911905 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:07.911788    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:07.915445 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:07.914498    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d6df0e9a-openshift-default-dockercfg-jz2cv\""
Sep 11 18:32:07.928424 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:07.928390    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"]
Sep 11 18:32:08.008456 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.008410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-socket\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.008643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.008484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-certs\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.008643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.008586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-token\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.008803 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.008638    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85dz7\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-kube-api-access-85dz7\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.008803 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.008665    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-credential-socket\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.008803 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.008760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-data\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.008951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.008827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d527e24-287f-4b31-b164-72e5e9cbbb42-istiod-ca-cert\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.008951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.008938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-podinfo\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.009046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.008965    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-envoy\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.110253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.110204    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-socket\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.110452 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.110284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-certs\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.110452 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.110345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-token\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.110452 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.110400    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85dz7\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-kube-api-access-85dz7\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.110452 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.110423    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-credential-socket\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.110663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.110467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-data\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.110663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.110512    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d527e24-287f-4b31-b164-72e5e9cbbb42-istiod-ca-cert\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.110663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.110567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-podinfo\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.110663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.110594    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-envoy\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.111866 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.111415    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-socket\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.111866 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.111484    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-certs\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.111866 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.111709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-data\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.112109 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.111973    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-credential-socket\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.112170 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.112114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d527e24-287f-4b31-b164-72e5e9cbbb42-istiod-ca-cert\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.115175 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.115128    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-podinfo\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.130099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.130068    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-envoy\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.132136 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.132108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-token\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.136521 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.136481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85dz7\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-kube-api-access-85dz7\") pod \"e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") " pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.228998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.228889    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:08.526005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.525975    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"]
Sep 11 18:32:08.529234 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:32:08.529204    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d527e24_287f_4b31_b164_72e5e9cbbb42.slice/crio-b9279910c46d17f73ee868b1d538a825449474af26f44e1e47601d1205c6ae99 WatchSource:0}: Error finding container b9279910c46d17f73ee868b1d538a825449474af26f44e1e47601d1205c6ae99: Status 404 returned error can't find the container with id b9279910c46d17f73ee868b1d538a825449474af26f44e1e47601d1205c6ae99
Sep 11 18:32:08.531912 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.531863    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:08.532013 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.531963    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:08.532013 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.532002    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:08.647429 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.647385    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" event={"ID":"6d527e24-287f-4b31-b164-72e5e9cbbb42","Type":"ContainerStarted","Data":"3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765"}
Sep 11 18:32:08.647926 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.647449    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" event={"ID":"6d527e24-287f-4b31-b164-72e5e9cbbb42","Type":"ContainerStarted","Data":"b9279910c46d17f73ee868b1d538a825449474af26f44e1e47601d1205c6ae99"}
Sep 11 18:32:08.668192 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:08.668128    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" podStartSLOduration=1.668108857 podStartE2EDuration="1.668108857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:08.666450307 +0000 UTC m=+991.701666978" watchObservedRunningTime="2026-09-11 18:32:08.668108857 +0000 UTC m=+991.703325527"
Sep 11 18:32:09.229993 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:09.229942    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:09.235125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:09.235098    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:09.651468 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:09.651383    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:09.652624 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:09.652603    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:14.385779 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.385735    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"]
Sep 11 18:32:14.408035 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.407995    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"]
Sep 11 18:32:14.408301 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.408279    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:14.416719 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.410944    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d6df0e9a\""
Sep 11 18:32:14.416719 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.411199    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d6df0e9a-dockercfg-4jnzz\""
Sep 11 18:32:14.609609 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.609561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkwrw\" (UniqueName: \"kubernetes.io/projected/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-kube-api-access-wkwrw\") pod \"payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw\" (UID: \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\") " pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:14.610091 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.610061    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-plugins-config-volume\") pod \"payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw\" (UID: \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\") " pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:14.711237 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.711186    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkwrw\" (UniqueName: \"kubernetes.io/projected/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-kube-api-access-wkwrw\") pod \"payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw\" (UID: \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\") " pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:14.711437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.711362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-plugins-config-volume\") pod \"payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw\" (UID: \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\") " pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:14.712423 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.712390    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-plugins-config-volume\") pod \"payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw\" (UID: \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\") " pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:14.721314 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.721284    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkwrw\" (UniqueName: \"kubernetes.io/projected/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-kube-api-access-wkwrw\") pod \"payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw\" (UID: \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\") " pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:14.783753 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:14.783715    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:15.126900 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:15.126739    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"]
Sep 11 18:32:15.751886 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:15.750918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw" event={"ID":"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9","Type":"ContainerStarted","Data":"f610728eff7db8945609a9ae53f0e8f629d67a5a1af6002a57ce7ea55b117ed1"}
Sep 11 18:32:15.751886 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:15.750994    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw" event={"ID":"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9","Type":"ContainerStarted","Data":"69cc36ace41952b0b1d4a3393b49fd2fe41bd1b5d49c80bc3fd117b510ed0549"}
Sep 11 18:32:15.751886 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:15.751815    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:15.776604 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:15.773278    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw" podStartSLOduration=1.773251855 podStartE2EDuration="1.773251855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:15.77010867 +0000 UTC m=+998.805325340" watchObservedRunningTime="2026-09-11 18:32:15.773251855 +0000 UTC m=+998.808468527"
Sep 11 18:32:16.346507 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.346414    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"]
Sep 11 18:32:16.350757 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.350721    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:16.356874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.356633    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-536a4d\""
Sep 11 18:32:16.357333 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.357140    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-536a4d-dockercfg-j6r6m\""
Sep 11 18:32:16.368376 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.368345    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"]
Sep 11 18:32:16.372183 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.372154    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca43c26-4def-43f1-8783-31474348edd9-plugins-config-volume\") pod \"payload-processing-e2e-autores-536a4d-7559ff877f-gcngv\" (UID: \"fca43c26-4def-43f1-8783-31474348edd9\") " pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:16.372495 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.372475    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9b9r\" (UniqueName: \"kubernetes.io/projected/fca43c26-4def-43f1-8783-31474348edd9-kube-api-access-f9b9r\") pod \"payload-processing-e2e-autores-536a4d-7559ff877f-gcngv\" (UID: \"fca43c26-4def-43f1-8783-31474348edd9\") " pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:16.482726 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.473932    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca43c26-4def-43f1-8783-31474348edd9-plugins-config-volume\") pod \"payload-processing-e2e-autores-536a4d-7559ff877f-gcngv\" (UID: \"fca43c26-4def-43f1-8783-31474348edd9\") " pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:16.482726 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.474095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9b9r\" (UniqueName: \"kubernetes.io/projected/fca43c26-4def-43f1-8783-31474348edd9-kube-api-access-f9b9r\") pod \"payload-processing-e2e-autores-536a4d-7559ff877f-gcngv\" (UID: \"fca43c26-4def-43f1-8783-31474348edd9\") " pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:16.482726 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.475904    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca43c26-4def-43f1-8783-31474348edd9-plugins-config-volume\") pod \"payload-processing-e2e-autores-536a4d-7559ff877f-gcngv\" (UID: \"fca43c26-4def-43f1-8783-31474348edd9\") " pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:16.503757 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.497803    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9b9r\" (UniqueName: \"kubernetes.io/projected/fca43c26-4def-43f1-8783-31474348edd9-kube-api-access-f9b9r\") pod \"payload-processing-e2e-autores-536a4d-7559ff877f-gcngv\" (UID: \"fca43c26-4def-43f1-8783-31474348edd9\") " pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:16.682152 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:16.682108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:17.198215 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:17.198173    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"]
Sep 11 18:32:17.641316 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:17.640318    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:32:17.642823 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:17.642798    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:32:17.821769 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:17.816762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv" event={"ID":"fca43c26-4def-43f1-8783-31474348edd9","Type":"ContainerStarted","Data":"dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9"}
Sep 11 18:32:17.821769 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:17.817046    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:17.821769 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:17.817079    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv" event={"ID":"fca43c26-4def-43f1-8783-31474348edd9","Type":"ContainerStarted","Data":"abefa7d9abab5f13864137bf3314ddddf5bb0c745e7ce362d37a43a531bdda66"}
Sep 11 18:32:17.870700 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:17.867603    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv" podStartSLOduration=1.867578322 podStartE2EDuration="1.867578322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:17.865477322 +0000 UTC m=+1000.900693993" watchObservedRunningTime="2026-09-11 18:32:17.867578322 +0000 UTC m=+1000.902794992"
Sep 11 18:32:18.648807 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:18.648772    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:32:21.843910 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.843879    2578 generic.go:356] "Generic (PLEG): container finished" podID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerID="1acf7b775eb9fec9290d2d22b6404da69510494684f8e7ddd7af40cd932e2815" exitCode=0
Sep 11 18:32:21.844349 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.843942    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" event={"ID":"bd6fc16f-db5c-4df4-8d13-5911feba2ee2","Type":"ContainerDied","Data":"1acf7b775eb9fec9290d2d22b6404da69510494684f8e7ddd7af40cd932e2815"}
Sep 11 18:32:21.844417 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.844389    2578 scope.go:117] "RemoveContainer" containerID="1acf7b775eb9fec9290d2d22b6404da69510494684f8e7ddd7af40cd932e2815"
Sep 11 18:32:21.845636 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.845606    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:21.845746 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.845729    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:21.845797 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.845746    2578 generic.go:356] "Generic (PLEG): container finished" podID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerID="45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1" exitCode=0
Sep 11 18:32:21.845797 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.845783    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:21.845866 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.845821    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" event={"ID":"44d90e55-a5ce-4d17-9c2f-8b46344193c5","Type":"ContainerDied","Data":"45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1"}
Sep 11 18:32:21.846185 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.846168    2578 scope.go:117] "RemoveContainer" containerID="45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1"
Sep 11 18:32:21.847294 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.847266    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:21.847384 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.847336    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:21.847431 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.847387    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:21.847577 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.847551    2578 generic.go:356] "Generic (PLEG): container finished" podID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerID="83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7" exitCode=0
Sep 11 18:32:21.847615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.847576    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" event={"ID":"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee","Type":"ContainerDied","Data":"83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7"}
Sep 11 18:32:21.847940 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.847925    2578 scope.go:117] "RemoveContainer" containerID="83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7"
Sep 11 18:32:21.849086 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.849058    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:21.849134 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.849123    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:21.849178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:21.849156    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:22.858822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:22.858785    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" event={"ID":"bd6fc16f-db5c-4df4-8d13-5911feba2ee2","Type":"ContainerStarted","Data":"d5d961fea269ef5f1258dd2c80a023a385e1b9043a643723e710a42f7cd0d205"}
Sep 11 18:32:22.867140 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:22.867101    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" event={"ID":"44d90e55-a5ce-4d17-9c2f-8b46344193c5","Type":"ContainerStarted","Data":"aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2"}
Sep 11 18:32:22.869964 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:22.869931    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" event={"ID":"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee","Type":"ContainerStarted","Data":"47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be"}
Sep 11 18:32:22.915381 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:22.915332    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:32:22.915587 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:22.915525    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 11 18:32:22.915664 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:22.915589    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 11 18:32:23.000110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:23.000067    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:32:23.000323 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:23.000293    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 11 18:32:23.000403 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:23.000347    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 11 18:32:23.078019 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:23.077928    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:32:23.084742 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:23.082799    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 18:32:23.084742 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:23.082862    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 18:32:23.915535 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:23.915487    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 11 18:32:23.916076 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:23.915582    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 11 18:32:24.000802 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.000763    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 11 18:32:24.001125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.001096    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 11 18:32:24.077203 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.077162    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 18:32:24.077496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.077467    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 18:32:24.702701 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.702079    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"]
Sep 11 18:32:24.720369 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.720330    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"]
Sep 11 18:32:24.720741 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.720651    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:24.734875 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.732695    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e8f201\""
Sep 11 18:32:24.734875 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.732961    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e8f201-dockercfg-5wgv6\""
Sep 11 18:32:24.841705 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.841151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8fdd\" (UniqueName: \"kubernetes.io/projected/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-kube-api-access-t8fdd\") pod \"payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln\" (UID: \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:24.841705 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.841260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln\" (UID: \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:24.916194 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.916150    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 11 18:32:24.916823 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.916789    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 11 18:32:24.942316 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.942268    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8fdd\" (UniqueName: \"kubernetes.io/projected/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-kube-api-access-t8fdd\") pod \"payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln\" (UID: \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:24.943086 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.943051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln\" (UID: \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:24.944247 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.944135    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln\" (UID: \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:24.958473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:24.958394    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8fdd\" (UniqueName: \"kubernetes.io/projected/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-kube-api-access-t8fdd\") pod \"payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln\" (UID: \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:25.000761 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.000721    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 11 18:32:25.001066 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.001029    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 11 18:32:25.063860 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.063819    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:25.078574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.078094    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 18:32:25.078574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.078192    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 18:32:25.646502 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.643862    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"]
Sep 11 18:32:25.676228 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:32:25.676192    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc293e3_7ffc_4966_8e4e_8c7a82fa84e8.slice/crio-0351b0f644a06ab6c09ac07543c13242f139ddc9c0dab1d3f9cb55bdd30c770a WatchSource:0}: Error finding container 0351b0f644a06ab6c09ac07543c13242f139ddc9c0dab1d3f9cb55bdd30c770a: Status 404 returned error can't find the container with id 0351b0f644a06ab6c09ac07543c13242f139ddc9c0dab1d3f9cb55bdd30c770a
Sep 11 18:32:25.915663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.915619    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln" event={"ID":"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8","Type":"ContainerStarted","Data":"c245b019e27a98f982e01a252f06610101ba77c2e0409c2006eb3473a16258fd"}
Sep 11 18:32:25.915892 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.915712    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln" event={"ID":"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8","Type":"ContainerStarted","Data":"0351b0f644a06ab6c09ac07543c13242f139ddc9c0dab1d3f9cb55bdd30c770a"}
Sep 11 18:32:25.916311 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.916282    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:25.916797 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.916772    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 11 18:32:25.916874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.916841    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 11 18:32:25.936728 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:25.936651    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln" podStartSLOduration=1.936630832 podStartE2EDuration="1.936630832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:25.932889479 +0000 UTC m=+1008.968106150" watchObservedRunningTime="2026-09-11 18:32:25.936630832 +0000 UTC m=+1008.971847503"
Sep 11 18:32:26.000374 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:26.000335    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 11 18:32:26.000661 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:26.000629    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 11 18:32:26.077783 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:26.077740    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 18:32:26.077978 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:26.077807    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 18:32:26.915002 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:26.914966    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 11 18:32:26.915350 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:26.915298    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 11 18:32:27.001192 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:27.001154    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 11 18:32:27.001803 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:27.001765    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 11 18:32:27.077315 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:27.077267    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 18:32:27.077607 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:27.077564    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 18:32:27.818862 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:27.818832    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:27.915271 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:27.915238    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 11 18:32:27.915541 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:27.915513    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 11 18:32:28.000297 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:28.000262    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 11 18:32:28.000564 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:28.000539    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 11 18:32:28.091738 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:28.091629    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 18:32:28.092295 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:28.092263    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 18:32:28.826290 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:28.826246    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:32:28.917215 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:28.916144    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 11 18:32:28.917215 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:28.916213    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 11 18:32:29.012246 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:29.012196    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 11 18:32:29.012517 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:29.012484    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 11 18:32:29.077544 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:29.077046    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 11 18:32:29.077544 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:29.077111    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 11 18:32:29.919236 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:29.919198    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:32:29.919622 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:29.919590    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:32:29.920455 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:29.920433    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:32:30.006067 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:30.006035    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:32:30.006463 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:30.006435    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:32:30.007319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:30.007299    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:32:30.082784 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:30.082648    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:32:30.083336 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:30.083281    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:32:30.084143 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:30.084123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:32:32.982862 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:32.982832    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerID="3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765" exitCode=0
Sep 11 18:32:32.983302 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:32.982893    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" event={"ID":"6d527e24-287f-4b31-b164-72e5e9cbbb42","Type":"ContainerDied","Data":"3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765"}
Sep 11 18:32:32.983346 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:32.983306    2578 scope.go:117] "RemoveContainer" containerID="3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765"
Sep 11 18:32:32.984497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:32.984469    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:32.984547 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:32.984538    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:32.984584 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:32.984573    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:33.990780 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:33.990747    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" event={"ID":"6d527e24-287f-4b31-b164-72e5e9cbbb42","Type":"ContainerStarted","Data":"001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403"}
Sep 11 18:32:34.229625 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:34.229585    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:34.230100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:34.230076    2578 patch_prober.go:28] interesting pod/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 11 18:32:34.230265 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:34.230243    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 11 18:32:35.230216 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:35.230161    2578 patch_prober.go:28] interesting pod/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 11 18:32:35.230758 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:35.230257    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 11 18:32:36.229554 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:36.229503    2578 patch_prober.go:28] interesting pod/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 11 18:32:36.229887 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:36.229581    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 11 18:32:36.928968 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:36.928916    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:32:37.230245 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:37.230131    2578 patch_prober.go:28] interesting pod/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 11 18:32:37.230245 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:37.230208    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 11 18:32:38.229609 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:38.229551    2578 patch_prober.go:28] interesting pod/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 11 18:32:38.230155 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:38.229627    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 11 18:32:39.229389 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:39.229342    2578 patch_prober.go:28] interesting pod/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 11 18:32:39.229709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:39.229657    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 11 18:32:39.652011 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:39.651922    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:40.234079 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.234046    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:40.235307 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.235281    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:40.587952 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.587856    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"]
Sep 11 18:32:40.591026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.591007    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"
Sep 11 18:32:40.593194 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.593172    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:32:40.602091 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.602058    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:32:40.604875 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.604846    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"]
Sep 11 18:32:40.669514 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.669418    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r\" (UID: \"96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"
Sep 11 18:32:40.771562 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.771524    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r\" (UID: \"96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"
Sep 11 18:32:40.772225 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.772204    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r\" (UID: \"96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"
Sep 11 18:32:40.905181 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:40.905089    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"
Sep 11 18:32:41.175989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:41.175960    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"]
Sep 11 18:32:41.178461 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:32:41.178427    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d87dd5_e97e_4fe0_8cb6_6b5f52c4bea3.slice/crio-5f25f7a9b8030fb4253692a8ac5d60054633cf6f09438012b9243a7b70a10c62 WatchSource:0}: Error finding container 5f25f7a9b8030fb4253692a8ac5d60054633cf6f09438012b9243a7b70a10c62: Status 404 returned error can't find the container with id 5f25f7a9b8030fb4253692a8ac5d60054633cf6f09438012b9243a7b70a10c62
Sep 11 18:32:42.045385 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:42.045347    2578 generic.go:356] "Generic (PLEG): container finished" podID="96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3" containerID="4060da1820382bff6378dfd3cdc206843f7afc726dc373484081a2364140b85b" exitCode=0
Sep 11 18:32:42.045806 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:42.045429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r" event={"ID":"96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3","Type":"ContainerDied","Data":"4060da1820382bff6378dfd3cdc206843f7afc726dc373484081a2364140b85b"}
Sep 11 18:32:42.045806 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:42.045497    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r" event={"ID":"96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3","Type":"ContainerStarted","Data":"5f25f7a9b8030fb4253692a8ac5d60054633cf6f09438012b9243a7b70a10c62"}
Sep 11 18:32:43.187848 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:43.187822    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"
Sep 11 18:32:43.297724 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:43.297667    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3-maas-api-service-ca\") pod \"96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3\" (UID: \"96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3\") "
Sep 11 18:32:43.298059 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:43.298034    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3" (UID: "96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:32:43.399398 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:43.399302    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:44.059363 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:44.059332    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"
Sep 11 18:32:44.059539 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:44.059363    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r" event={"ID":"96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3","Type":"ContainerDied","Data":"5f25f7a9b8030fb4253692a8ac5d60054633cf6f09438012b9243a7b70a10c62"}
Sep 11 18:32:44.059539 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:44.059412    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f25f7a9b8030fb4253692a8ac5d60054633cf6f09438012b9243a7b70a10c62"
Sep 11 18:32:51.857329 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:51.857294    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"]
Sep 11 18:32:51.857945 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:51.857503    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw" podUID="22e3d0d4-639a-4162-aa5a-d5e8929f0cd9" containerName="payload-processing" containerID="cri-o://f610728eff7db8945609a9ae53f0e8f629d67a5a1af6002a57ce7ea55b117ed1" gracePeriod=30
Sep 11 18:32:52.017371 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.017328    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"]
Sep 11 18:32:52.019321 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.019292    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6df0e9a-d4c7r"]
Sep 11 18:32:52.103116 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.103087    2578 generic.go:356] "Generic (PLEG): container finished" podID="22e3d0d4-639a-4162-aa5a-d5e8929f0cd9" containerID="f610728eff7db8945609a9ae53f0e8f629d67a5a1af6002a57ce7ea55b117ed1" exitCode=0
Sep 11 18:32:52.103270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.103154    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw" event={"ID":"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9","Type":"ContainerDied","Data":"f610728eff7db8945609a9ae53f0e8f629d67a5a1af6002a57ce7ea55b117ed1"}
Sep 11 18:32:52.121978 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.121910    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:52.161347 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.161313    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"]
Sep 11 18:32:52.161544 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.161521    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy" containerID="cri-o://001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403" gracePeriod=30
Sep 11 18:32:52.188623 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.188591    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkwrw\" (UniqueName: \"kubernetes.io/projected/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-kube-api-access-wkwrw\") pod \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\" (UID: \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\") "
Sep 11 18:32:52.188797 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.188663    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-plugins-config-volume\") pod \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\" (UID: \"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9\") "
Sep 11 18:32:52.189087 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.189062    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "22e3d0d4-639a-4162-aa5a-d5e8929f0cd9" (UID: "22e3d0d4-639a-4162-aa5a-d5e8929f0cd9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:32:52.190940 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.190917    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-kube-api-access-wkwrw" (OuterVolumeSpecName: "kube-api-access-wkwrw") pod "22e3d0d4-639a-4162-aa5a-d5e8929f0cd9" (UID: "22e3d0d4-639a-4162-aa5a-d5e8929f0cd9"). InnerVolumeSpecName "kube-api-access-wkwrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:32:52.289506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.289466    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkwrw\" (UniqueName: \"kubernetes.io/projected/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-kube-api-access-wkwrw\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:52.289506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:52.289500    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:53.109947 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.109916    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"
Sep 11 18:32:53.110383 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.109964    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw" event={"ID":"22e3d0d4-639a-4162-aa5a-d5e8929f0cd9","Type":"ContainerDied","Data":"69cc36ace41952b0b1d4a3393b49fd2fe41bd1b5d49c80bc3fd117b510ed0549"}
Sep 11 18:32:53.110383 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.110028    2578 scope.go:117] "RemoveContainer" containerID="f610728eff7db8945609a9ae53f0e8f629d67a5a1af6002a57ce7ea55b117ed1"
Sep 11 18:32:53.149966 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.149933    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"]
Sep 11 18:32:53.158775 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.158739    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6df0e9a-5dcf57f68f-ckfmw"]
Sep 11 18:32:53.160825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.160803    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"]
Sep 11 18:32:53.161306 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.161280    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3" containerName="revoke-keys"
Sep 11 18:32:53.161306 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.161304    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3" containerName="revoke-keys"
Sep 11 18:32:53.161444 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.161323    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e3d0d4-639a-4162-aa5a-d5e8929f0cd9" containerName="payload-processing"
Sep 11 18:32:53.161444 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.161332    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e3d0d4-639a-4162-aa5a-d5e8929f0cd9" containerName="payload-processing"
Sep 11 18:32:53.161444 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.161414    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="22e3d0d4-639a-4162-aa5a-d5e8929f0cd9" containerName="payload-processing"
Sep 11 18:32:53.161444 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.161424    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3" containerName="revoke-keys"
Sep 11 18:32:53.164068 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.164053    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.166347 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.166327    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-414d08da-gw-openshift-default-dockercfg-gznr7\""
Sep 11 18:32:53.177783 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.177757    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"]
Sep 11 18:32:53.299902 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.299846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-certs\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.300100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.299930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-socket\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.300100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.299989    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-podinfo\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.300100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.300016    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-token\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.300100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.300069    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-credential-socket\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.300100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.300096    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-data\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.300353 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.300147    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-envoy\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.300353 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.300210    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istiod-ca-cert\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.300353 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.300256    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clthd\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-kube-api-access-clthd\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.401378 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.401284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-certs\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.402201 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.402175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-socket\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.402935 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.402913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-podinfo\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.406793 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.406764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-token\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.408137 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.408109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-credential-socket\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.408737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.408717    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-data\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.409211 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.409192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-envoy\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.409708 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.408651    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-credential-socket\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.409864 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.402807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-socket\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.409979 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.409113    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-data\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.410078 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.402077    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-certs\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.410175 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.409729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istiod-ca-cert\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.410322 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.410305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clthd\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-kube-api-access-clthd\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.410494 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.410383    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istiod-ca-cert\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.413324 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.413297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-envoy\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.414061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.413539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-podinfo\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.417756 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.417732    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-token\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.419609 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.419582    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22e3d0d4-639a-4162-aa5a-d5e8929f0cd9" path="/var/lib/kubelet/pods/22e3d0d4-639a-4162-aa5a-d5e8929f0cd9/volumes"
Sep 11 18:32:53.419736 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.419717    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clthd\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-kube-api-access-clthd\") pod \"e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") " pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.419998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.419981    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3" path="/var/lib/kubelet/pods/96d87dd5-e97e-4fe0-8cb6-6b5f52c4bea3/volumes"
Sep 11 18:32:53.474792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.474754    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:53.756154 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.756123    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"]
Sep 11 18:32:53.758757 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:32:53.758728    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a3206e7_66ee_41ae_83a6_0c95e53b3e09.slice/crio-88f428c76bad4e5071652aac3fb0645df4a94c813191b6d15ee5cc51f4098920 WatchSource:0}: Error finding container 88f428c76bad4e5071652aac3fb0645df4a94c813191b6d15ee5cc51f4098920: Status 404 returned error can't find the container with id 88f428c76bad4e5071652aac3fb0645df4a94c813191b6d15ee5cc51f4098920
Sep 11 18:32:53.761265 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.761233    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:53.761330 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.761302    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:53.761362 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:53.761338    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:32:54.123757 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:54.122795    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" event={"ID":"5a3206e7-66ee-41ae-83a6-0c95e53b3e09","Type":"ContainerStarted","Data":"8c97d0356d247ee8e2557aaddad4f8e2cbeca2e849f176c6e98e2faaa0e1b5fe"}
Sep 11 18:32:54.123757 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:54.122882    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" event={"ID":"5a3206e7-66ee-41ae-83a6-0c95e53b3e09","Type":"ContainerStarted","Data":"88f428c76bad4e5071652aac3fb0645df4a94c813191b6d15ee5cc51f4098920"}
Sep 11 18:32:54.147323 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:54.147269    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" podStartSLOduration=1.147252082 podStartE2EDuration="1.147252082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:54.14428159 +0000 UTC m=+1037.179498271" watchObservedRunningTime="2026-09-11 18:32:54.147252082 +0000 UTC m=+1037.182468751"
Sep 11 18:32:54.475365 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:54.475334    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:54.480208 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:54.480183    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:55.128390 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:55.128350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:55.129306 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:55.129285    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:32:55.237220 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:55.237184    2578 patch_prober.go:28] interesting pod/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:32:55.237405 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:55.237244    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:32:57.426289 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.426230    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:57.553445 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553416    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-certs\") pod \"6d527e24-287f-4b31-b164-72e5e9cbbb42\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") "
Sep 11 18:32:57.553646 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553480    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85dz7\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-kube-api-access-85dz7\") pod \"6d527e24-287f-4b31-b164-72e5e9cbbb42\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") "
Sep 11 18:32:57.553646 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553522    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-data\") pod \"6d527e24-287f-4b31-b164-72e5e9cbbb42\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") "
Sep 11 18:32:57.553803 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553668    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-podinfo\") pod \"6d527e24-287f-4b31-b164-72e5e9cbbb42\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") "
Sep 11 18:32:57.553803 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553761    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-credential-socket\") pod \"6d527e24-287f-4b31-b164-72e5e9cbbb42\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") "
Sep 11 18:32:57.553803 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553775    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d527e24-287f-4b31-b164-72e5e9cbbb42" (UID: "6d527e24-287f-4b31-b164-72e5e9cbbb42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:32:57.553962 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553786    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-socket\") pod \"6d527e24-287f-4b31-b164-72e5e9cbbb42\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") "
Sep 11 18:32:57.553962 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553862    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-token\") pod \"6d527e24-287f-4b31-b164-72e5e9cbbb42\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") "
Sep 11 18:32:57.553962 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553883    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d527e24-287f-4b31-b164-72e5e9cbbb42" (UID: "6d527e24-287f-4b31-b164-72e5e9cbbb42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:32:57.553962 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553933    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d527e24-287f-4b31-b164-72e5e9cbbb42-istiod-ca-cert\") pod \"6d527e24-287f-4b31-b164-72e5e9cbbb42\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") "
Sep 11 18:32:57.554157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553962    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-envoy\") pod \"6d527e24-287f-4b31-b164-72e5e9cbbb42\" (UID: \"6d527e24-287f-4b31-b164-72e5e9cbbb42\") "
Sep 11 18:32:57.554157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.553988    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d527e24-287f-4b31-b164-72e5e9cbbb42" (UID: "6d527e24-287f-4b31-b164-72e5e9cbbb42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:32:57.554157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.554019    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d527e24-287f-4b31-b164-72e5e9cbbb42" (UID: "6d527e24-287f-4b31-b164-72e5e9cbbb42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:32:57.554310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.554286    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d527e24-287f-4b31-b164-72e5e9cbbb42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d527e24-287f-4b31-b164-72e5e9cbbb42" (UID: "6d527e24-287f-4b31-b164-72e5e9cbbb42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:32:57.554414 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.554386    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d527e24-287f-4b31-b164-72e5e9cbbb42-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:57.554414 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.554409    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:57.554504 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.554423    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:57.554504 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.554436    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:57.554504 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.554449    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:57.555999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.555973    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-kube-api-access-85dz7" (OuterVolumeSpecName: "kube-api-access-85dz7") pod "6d527e24-287f-4b31-b164-72e5e9cbbb42" (UID: "6d527e24-287f-4b31-b164-72e5e9cbbb42"). InnerVolumeSpecName "kube-api-access-85dz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:32:57.555999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.555990    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d527e24-287f-4b31-b164-72e5e9cbbb42" (UID: "6d527e24-287f-4b31-b164-72e5e9cbbb42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:32:57.556242 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.556212    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d527e24-287f-4b31-b164-72e5e9cbbb42" (UID: "6d527e24-287f-4b31-b164-72e5e9cbbb42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:32:57.556321 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.556298    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d527e24-287f-4b31-b164-72e5e9cbbb42" (UID: "6d527e24-287f-4b31-b164-72e5e9cbbb42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:32:57.655089 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.655051    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:57.655089 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.655081    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85dz7\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-kube-api-access-85dz7\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:57.655089 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.655096    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:57.655349 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:57.655110    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d527e24-287f-4b31-b164-72e5e9cbbb42-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:32:58.150315 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.150267    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerID="001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403" exitCode=0
Sep 11 18:32:58.150826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.150544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" event={"ID":"6d527e24-287f-4b31-b164-72e5e9cbbb42","Type":"ContainerDied","Data":"001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403"}
Sep 11 18:32:58.151203 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.151144    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb" event={"ID":"6d527e24-287f-4b31-b164-72e5e9cbbb42","Type":"ContainerDied","Data":"b9279910c46d17f73ee868b1d538a825449474af26f44e1e47601d1205c6ae99"}
Sep 11 18:32:58.151329 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.150749    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"
Sep 11 18:32:58.151565 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.151185    2578 scope.go:117] "RemoveContainer" containerID="001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403"
Sep 11 18:32:58.170842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.170822    2578 scope.go:117] "RemoveContainer" containerID="3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765"
Sep 11 18:32:58.180000 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.179977    2578 scope.go:117] "RemoveContainer" containerID="001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403"
Sep 11 18:32:58.180341 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:32:58.180319    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403\": container with ID starting with 001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403 not found: ID does not exist" containerID="001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403"
Sep 11 18:32:58.180422 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.180349    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403"} err="failed to get container status \"001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403\": rpc error: code = NotFound desc = could not find container \"001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403\": container with ID starting with 001091af09b82a146cdbfc13c0c281c95eb3860ba473077f7bb8ca3332968403 not found: ID does not exist"
Sep 11 18:32:58.180422 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.180381    2578 scope.go:117] "RemoveContainer" containerID="3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765"
Sep 11 18:32:58.180646 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:32:58.180631    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765\": container with ID starting with 3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765 not found: ID does not exist" containerID="3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765"
Sep 11 18:32:58.180732 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.180649    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765"} err="failed to get container status \"3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765\": rpc error: code = NotFound desc = could not find container \"3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765\": container with ID starting with 3b53cacde58df0ec6e8f9088f2766cc84834d50326f4e15953c22775823b2765 not found: ID does not exist"
Sep 11 18:32:58.205010 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.204982    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"]
Sep 11 18:32:58.209482 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.209458    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d6df0e9a-openshift-default-b8948f754-czzgb"]
Sep 11 18:32:58.358354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.358316    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"]
Sep 11 18:32:58.358737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.358724    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy"
Sep 11 18:32:58.358795 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.358738    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy"
Sep 11 18:32:58.358795 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.358752    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy"
Sep 11 18:32:58.358795 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.358758    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy"
Sep 11 18:32:58.358883 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.358824    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy"
Sep 11 18:32:58.358883 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.358833    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" containerName="istio-proxy"
Sep 11 18:32:58.361806 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.361788    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:32:58.364760 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.364739    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-414d08da\""
Sep 11 18:32:58.374965 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.374941    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"]
Sep 11 18:32:58.463201 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.463165    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26smq\" (UniqueName: \"kubernetes.io/projected/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-kube-api-access-26smq\") pod \"payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f\" (UID: \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:32:58.463612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.463220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f\" (UID: \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:32:58.507514 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.507470    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"]
Sep 11 18:32:58.510882 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.510853    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:32:58.514321 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.514292    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-414d08da-dockercfg-hppzp\""
Sep 11 18:32:58.520816 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.520785    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"]
Sep 11 18:32:58.564084 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.564038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26smq\" (UniqueName: \"kubernetes.io/projected/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-kube-api-access-26smq\") pod \"payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f\" (UID: \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:32:58.564273 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.564155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f\" (UID: \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:32:58.564889 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.564867    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f\" (UID: \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:32:58.577178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.577156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26smq\" (UniqueName: \"kubernetes.io/projected/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-kube-api-access-26smq\") pod \"payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f\" (UID: \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:32:58.665178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.665146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zchkd\" (UniqueName: \"kubernetes.io/projected/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-kube-api-access-zchkd\") pod \"payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4\" (UID: \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\") " pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:32:58.665339 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.665242    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4\" (UID: \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\") " pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:32:58.671935 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.671905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:32:58.766373 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.766340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4\" (UID: \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\") " pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:32:58.766539 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.766406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zchkd\" (UniqueName: \"kubernetes.io/projected/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-kube-api-access-zchkd\") pod \"payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4\" (UID: \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\") " pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:32:58.767149 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.767124    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4\" (UID: \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\") " pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:32:58.776087 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.776058    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zchkd\" (UniqueName: \"kubernetes.io/projected/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-kube-api-access-zchkd\") pod \"payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4\" (UID: \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\") " pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:32:58.834035 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.833997    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:32:58.945544 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:58.945507    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"]
Sep 11 18:32:58.946858 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:32:58.946831    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c22496_8b28_4717_b96c_d77fe4bfc1f8.slice/crio-6f09fe47764156c963614d317cba02787fb8dc35c89fba76c7c59ab027fed043 WatchSource:0}: Error finding container 6f09fe47764156c963614d317cba02787fb8dc35c89fba76c7c59ab027fed043: Status 404 returned error can't find the container with id 6f09fe47764156c963614d317cba02787fb8dc35c89fba76c7c59ab027fed043
Sep 11 18:32:59.112855 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:59.112824    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"]
Sep 11 18:32:59.114313 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:32:59.114288    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d337c0f_905d_4ad4_b58f_52f8f2ca52bd.slice/crio-0ad3c884120674c523e7b33f0f8c081b70271156d13f735ece1a1b6f867b4708 WatchSource:0}: Error finding container 0ad3c884120674c523e7b33f0f8c081b70271156d13f735ece1a1b6f867b4708: Status 404 returned error can't find the container with id 0ad3c884120674c523e7b33f0f8c081b70271156d13f735ece1a1b6f867b4708
Sep 11 18:32:59.159125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:59.159095    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f" event={"ID":"e8c22496-8b28-4717-b96c-d77fe4bfc1f8","Type":"ContainerStarted","Data":"532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85"}
Sep 11 18:32:59.159267 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:59.159143    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f" event={"ID":"e8c22496-8b28-4717-b96c-d77fe4bfc1f8","Type":"ContainerStarted","Data":"6f09fe47764156c963614d317cba02787fb8dc35c89fba76c7c59ab027fed043"}
Sep 11 18:32:59.159267 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:59.159229    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:32:59.160477 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:59.160448    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4" event={"ID":"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd","Type":"ContainerStarted","Data":"0ad3c884120674c523e7b33f0f8c081b70271156d13f735ece1a1b6f867b4708"}
Sep 11 18:32:59.178105 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:59.178051    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f" podStartSLOduration=1.178035518 podStartE2EDuration="1.178035518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:59.175998154 +0000 UTC m=+1042.211214823" watchObservedRunningTime="2026-09-11 18:32:59.178035518 +0000 UTC m=+1042.213252200"
Sep 11 18:32:59.410887 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:32:59.410861    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d527e24-287f-4b31-b164-72e5e9cbbb42" path="/var/lib/kubelet/pods/6d527e24-287f-4b31-b164-72e5e9cbbb42/volumes"
Sep 11 18:33:00.167802 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:00.167762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4" event={"ID":"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd","Type":"ContainerStarted","Data":"3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3"}
Sep 11 18:33:00.185455 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:00.185399    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4" podStartSLOduration=2.185383114 podStartE2EDuration="2.185383114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:33:00.183764269 +0000 UTC m=+1043.218980952" watchObservedRunningTime="2026-09-11 18:33:00.185383114 +0000 UTC m=+1043.220599784"
Sep 11 18:33:01.173883 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:01.173850    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:33:10.170659 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:10.170629    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:33:12.180859 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:12.180824    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:33:16.277778 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:16.277659    2578 generic.go:356] "Generic (PLEG): container finished" podID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerID="8c97d0356d247ee8e2557aaddad4f8e2cbeca2e849f176c6e98e2faaa0e1b5fe" exitCode=0
Sep 11 18:33:16.277778 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:16.277733    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" event={"ID":"5a3206e7-66ee-41ae-83a6-0c95e53b3e09","Type":"ContainerDied","Data":"8c97d0356d247ee8e2557aaddad4f8e2cbeca2e849f176c6e98e2faaa0e1b5fe"}
Sep 11 18:33:16.278316 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:16.278273    2578 scope.go:117] "RemoveContainer" containerID="8c97d0356d247ee8e2557aaddad4f8e2cbeca2e849f176c6e98e2faaa0e1b5fe"
Sep 11 18:33:16.279455 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:16.279430    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:33:16.279525 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:16.279497    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:33:16.279567 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:16.279542    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:33:17.284742 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:17.284703    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" event={"ID":"5a3206e7-66ee-41ae-83a6-0c95e53b3e09","Type":"ContainerStarted","Data":"16786729dfd9f070268ef937e72f2e26a9f2fb9c68a66ebf3ad18e2b3f36c0d4"}
Sep 11 18:33:17.475778 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:17.475746    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:33:17.476009 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:17.475990    2578 patch_prober.go:28] interesting pod/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 18:33:17.476067 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:17.476034    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 18:33:18.475585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:18.475544    2578 patch_prober.go:28] interesting pod/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 18:33:18.476030 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:18.475603    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 18:33:19.475931 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:19.475894    2578 patch_prober.go:28] interesting pod/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 18:33:19.476320 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:19.475952    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 18:33:20.476047 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.476010    2578 patch_prober.go:28] interesting pod/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 18:33:20.476439 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.476067    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 18:33:20.559476 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.559441    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"]
Sep 11 18:33:20.561951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.561934    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"
Sep 11 18:33:20.564133 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.564112    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:33:20.564195 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.564182    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:33:20.570615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.570586    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"]
Sep 11 18:33:20.686001 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.685959    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d11597-8908-4239-ac88-52100b7ced4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-414d08da-jph5w\" (UID: \"c0d11597-8908-4239-ac88-52100b7ced4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"
Sep 11 18:33:20.787318 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.787220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d11597-8908-4239-ac88-52100b7ced4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-414d08da-jph5w\" (UID: \"c0d11597-8908-4239-ac88-52100b7ced4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"
Sep 11 18:33:20.787969 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.787942    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d11597-8908-4239-ac88-52100b7ced4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-414d08da-jph5w\" (UID: \"c0d11597-8908-4239-ac88-52100b7ced4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"
Sep 11 18:33:20.874548 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:20.874512    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"
Sep 11 18:33:21.168471 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:21.168446    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"]
Sep 11 18:33:21.170174 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:33:21.170141    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d11597_8908_4239_ac88_52100b7ced4c.slice/crio-2bdc93d06da65c34410a89311d3802580a2553b1fa7852c934a3f022f199484b WatchSource:0}: Error finding container 2bdc93d06da65c34410a89311d3802580a2553b1fa7852c934a3f022f199484b: Status 404 returned error can't find the container with id 2bdc93d06da65c34410a89311d3802580a2553b1fa7852c934a3f022f199484b
Sep 11 18:33:21.306344 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:21.306308    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w" event={"ID":"c0d11597-8908-4239-ac88-52100b7ced4c","Type":"ContainerStarted","Data":"2bdc93d06da65c34410a89311d3802580a2553b1fa7852c934a3f022f199484b"}
Sep 11 18:33:21.475163 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:21.475137    2578 patch_prober.go:28] interesting pod/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 18:33:21.475289 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:21.475183    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 18:33:22.316433 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:22.316328    2578 generic.go:356] "Generic (PLEG): container finished" podID="c0d11597-8908-4239-ac88-52100b7ced4c" containerID="076cac05e38693c032433a8aed585ff395cbea2d2fc269d2faad37f4bfe5ac0b" exitCode=0
Sep 11 18:33:22.316940 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:22.316511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w" event={"ID":"c0d11597-8908-4239-ac88-52100b7ced4c","Type":"ContainerDied","Data":"076cac05e38693c032433a8aed585ff395cbea2d2fc269d2faad37f4bfe5ac0b"}
Sep 11 18:33:22.475569 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:22.475532    2578 patch_prober.go:28] interesting pod/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 11 18:33:22.475764 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:22.475597    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 11 18:33:23.485179 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:23.485149    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"
Sep 11 18:33:23.639704 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:23.639590    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d11597-8908-4239-ac88-52100b7ced4c-maas-api-service-ca\") pod \"c0d11597-8908-4239-ac88-52100b7ced4c\" (UID: \"c0d11597-8908-4239-ac88-52100b7ced4c\") "
Sep 11 18:33:23.640068 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:23.640037    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d11597-8908-4239-ac88-52100b7ced4c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0d11597-8908-4239-ac88-52100b7ced4c" (UID: "c0d11597-8908-4239-ac88-52100b7ced4c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:33:23.669772 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:23.669734    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:33:23.670175 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:23.670155    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:33:23.672438 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:23.672411    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:33:23.741425 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:23.741388    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d11597-8908-4239-ac88-52100b7ced4c-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:24.331746 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:24.331710    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w" event={"ID":"c0d11597-8908-4239-ac88-52100b7ced4c","Type":"ContainerDied","Data":"2bdc93d06da65c34410a89311d3802580a2553b1fa7852c934a3f022f199484b"}
Sep 11 18:33:24.331957 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:24.331774    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bdc93d06da65c34410a89311d3802580a2553b1fa7852c934a3f022f199484b"
Sep 11 18:33:24.331957 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:24.331725    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"
Sep 11 18:33:26.911585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:26.911550    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"]
Sep 11 18:33:26.912118 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:26.911784    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4" podUID="4d337c0f-905d-4ad4-b58f-52f8f2ca52bd" containerName="payload-processing" containerID="cri-o://3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3" gracePeriod=30
Sep 11 18:33:26.921210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:26.921177    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"]
Sep 11 18:33:26.921659 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:26.921631    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f" podUID="e8c22496-8b28-4717-b96c-d77fe4bfc1f8" containerName="payload-pre-processing" containerID="cri-o://532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85" gracePeriod=30
Sep 11 18:33:27.071800 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.071668    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"]
Sep 11 18:33:27.075321 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.075288    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-414d08da-jph5w"]
Sep 11 18:33:27.228025 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.221848    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"]
Sep 11 18:33:27.228025 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.222141    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy" containerID="cri-o://16786729dfd9f070268ef937e72f2e26a9f2fb9c68a66ebf3ad18e2b3f36c0d4" gracePeriod=30
Sep 11 18:33:27.246838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.246806    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:33:27.251515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.251492    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:33:27.354558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.354520    2578 generic.go:356] "Generic (PLEG): container finished" podID="e8c22496-8b28-4717-b96c-d77fe4bfc1f8" containerID="532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85" exitCode=0
Sep 11 18:33:27.354790 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.354591    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"
Sep 11 18:33:27.354790 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.354603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f" event={"ID":"e8c22496-8b28-4717-b96c-d77fe4bfc1f8","Type":"ContainerDied","Data":"532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85"}
Sep 11 18:33:27.354790 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.354716    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f" event={"ID":"e8c22496-8b28-4717-b96c-d77fe4bfc1f8","Type":"ContainerDied","Data":"6f09fe47764156c963614d317cba02787fb8dc35c89fba76c7c59ab027fed043"}
Sep 11 18:33:27.354790 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.354754    2578 scope.go:117] "RemoveContainer" containerID="532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85"
Sep 11 18:33:27.356008 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.355985    2578 generic.go:356] "Generic (PLEG): container finished" podID="4d337c0f-905d-4ad4-b58f-52f8f2ca52bd" containerID="3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3" exitCode=0
Sep 11 18:33:27.356112 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.356036    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"
Sep 11 18:33:27.356112 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.356072    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4" event={"ID":"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd","Type":"ContainerDied","Data":"3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3"}
Sep 11 18:33:27.356191 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.356115    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4" event={"ID":"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd","Type":"ContainerDied","Data":"0ad3c884120674c523e7b33f0f8c081b70271156d13f735ece1a1b6f867b4708"}
Sep 11 18:33:27.363811 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.363791    2578 scope.go:117] "RemoveContainer" containerID="532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85"
Sep 11 18:33:27.364108 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:33:27.364088    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85\": container with ID starting with 532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85 not found: ID does not exist" containerID="532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85"
Sep 11 18:33:27.364191 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.364122    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85"} err="failed to get container status \"532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85\": rpc error: code = NotFound desc = could not find container \"532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85\": container with ID starting with 532c37e622a6dc69c511d8812d3edc474e61d6621c1ff4bc96209e88cd8ede85 not found: ID does not exist"
Sep 11 18:33:27.364191 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.364146    2578 scope.go:117] "RemoveContainer" containerID="3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3"
Sep 11 18:33:27.372945 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.372921    2578 scope.go:117] "RemoveContainer" containerID="3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3"
Sep 11 18:33:27.373262 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:33:27.373246    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3\": container with ID starting with 3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3 not found: ID does not exist" containerID="3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3"
Sep 11 18:33:27.373302 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.373270    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3"} err="failed to get container status \"3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3\": rpc error: code = NotFound desc = could not find container \"3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3\": container with ID starting with 3814045f3404be9505891c700d1c590934a9d14226e33eef9daf0b11ebdb9ae3 not found: ID does not exist"
Sep 11 18:33:27.389663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.389625    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-plugins-config-volume\") pod \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\" (UID: \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\") "
Sep 11 18:33:27.389822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.389672    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-plugins-config-volume\") pod \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\" (UID: \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\") "
Sep 11 18:33:27.389822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.389718    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26smq\" (UniqueName: \"kubernetes.io/projected/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-kube-api-access-26smq\") pod \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\" (UID: \"e8c22496-8b28-4717-b96c-d77fe4bfc1f8\") "
Sep 11 18:33:27.389948 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.389830    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zchkd\" (UniqueName: \"kubernetes.io/projected/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-kube-api-access-zchkd\") pod \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\" (UID: \"4d337c0f-905d-4ad4-b58f-52f8f2ca52bd\") "
Sep 11 18:33:27.390115 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.390080    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d337c0f-905d-4ad4-b58f-52f8f2ca52bd" (UID: "4d337c0f-905d-4ad4-b58f-52f8f2ca52bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:33:27.390222 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.390122    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e8c22496-8b28-4717-b96c-d77fe4bfc1f8" (UID: "e8c22496-8b28-4717-b96c-d77fe4bfc1f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:33:27.391930 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.391906    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-kube-api-access-zchkd" (OuterVolumeSpecName: "kube-api-access-zchkd") pod "4d337c0f-905d-4ad4-b58f-52f8f2ca52bd" (UID: "4d337c0f-905d-4ad4-b58f-52f8f2ca52bd"). InnerVolumeSpecName "kube-api-access-zchkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:33:27.392035 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.391998    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-kube-api-access-26smq" (OuterVolumeSpecName: "kube-api-access-26smq") pod "e8c22496-8b28-4717-b96c-d77fe4bfc1f8" (UID: "e8c22496-8b28-4717-b96c-d77fe4bfc1f8"). InnerVolumeSpecName "kube-api-access-26smq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:33:27.411272 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.411236    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0d11597-8908-4239-ac88-52100b7ced4c" path="/var/lib/kubelet/pods/c0d11597-8908-4239-ac88-52100b7ced4c/volumes"
Sep 11 18:33:27.490989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.490890    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:27.490989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.490932    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:27.490989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.490944    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26smq\" (UniqueName: \"kubernetes.io/projected/e8c22496-8b28-4717-b96c-d77fe4bfc1f8-kube-api-access-26smq\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:27.490989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.490953    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zchkd\" (UniqueName: \"kubernetes.io/projected/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd-kube-api-access-zchkd\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:27.676327 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.676289    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"]
Sep 11 18:33:27.685298 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.685266    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-414d08da-5899869ddb-lq79f"]
Sep 11 18:33:27.696224 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.696199    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"]
Sep 11 18:33:27.701444 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.701415    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-414d08da-86888b4954-2jpw4"]
Sep 11 18:33:27.919110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919075    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"]
Sep 11 18:33:27.919724 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919701    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d11597-8908-4239-ac88-52100b7ced4c" containerName="revoke-keys"
Sep 11 18:33:27.919769 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919727    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d11597-8908-4239-ac88-52100b7ced4c" containerName="revoke-keys"
Sep 11 18:33:27.919800 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919776    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8c22496-8b28-4717-b96c-d77fe4bfc1f8" containerName="payload-pre-processing"
Sep 11 18:33:27.919800 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919786    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8c22496-8b28-4717-b96c-d77fe4bfc1f8" containerName="payload-pre-processing"
Sep 11 18:33:27.919879 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919800    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d337c0f-905d-4ad4-b58f-52f8f2ca52bd" containerName="payload-processing"
Sep 11 18:33:27.919879 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919809    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d337c0f-905d-4ad4-b58f-52f8f2ca52bd" containerName="payload-processing"
Sep 11 18:33:27.919939 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919916    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8c22496-8b28-4717-b96c-d77fe4bfc1f8" containerName="payload-pre-processing"
Sep 11 18:33:27.919939 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919933    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d11597-8908-4239-ac88-52100b7ced4c" containerName="revoke-keys"
Sep 11 18:33:27.919996 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.919944    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d337c0f-905d-4ad4-b58f-52f8f2ca52bd" containerName="payload-processing"
Sep 11 18:33:27.931733 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.931699    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:27.934052 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.934026    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"]
Sep 11 18:33:27.934308 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:27.934289    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f8e161a6-openshift-default-dockercfg-crtcq\""
Sep 11 18:33:28.098152 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.098114    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-token\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.098152 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.098154    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-credential-socket\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.098382 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.098180    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-socket\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.098382 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.098268    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llw7w\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-kube-api-access-llw7w\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.098382 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.098308    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-data\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.098382 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.098362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-envoy\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.098522 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.098448    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istiod-ca-cert\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.098522 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.098491    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-certs\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.098522 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.098518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-podinfo\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.199605 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.199508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-podinfo\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.199605 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.199564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-token\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.199605 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.199594    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-credential-socket\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.199955 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.199630    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-socket\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.199955 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.199670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llw7w\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-kube-api-access-llw7w\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.199955 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.199782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-data\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.199955 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.199856    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-envoy\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.200167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.199968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istiod-ca-cert\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.200167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.200023    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-certs\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.200167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.200107    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-credential-socket\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.200167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.200131    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-socket\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.200608 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.200187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-data\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.200608 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.200360    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-certs\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.200695 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.200638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istiod-ca-cert\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.202085 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.202062    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-envoy\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.202354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.202337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-podinfo\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.208159 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.208139    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-token\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.208416 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.208385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llw7w\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-kube-api-access-llw7w\") pod \"e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") " pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.245127 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.245090    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:28.537376 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.537349    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"]
Sep 11 18:33:28.539177 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:33:28.539139    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c723f6f_e462_4e0b_b53d_f2e5ec5ae6d1.slice/crio-cf83264668c6c0f5e4c78a745f76b813802b0ea633e0380b86bd854cbe73cb7c WatchSource:0}: Error finding container cf83264668c6c0f5e4c78a745f76b813802b0ea633e0380b86bd854cbe73cb7c: Status 404 returned error can't find the container with id cf83264668c6c0f5e4c78a745f76b813802b0ea633e0380b86bd854cbe73cb7c
Sep 11 18:33:28.541402 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.541368    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:33:28.541511 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.541446    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:33:28.541511 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:28.541485    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:33:29.371461 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:29.371426    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" event={"ID":"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1","Type":"ContainerStarted","Data":"eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333"}
Sep 11 18:33:29.371934 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:29.371478    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" event={"ID":"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1","Type":"ContainerStarted","Data":"cf83264668c6c0f5e4c78a745f76b813802b0ea633e0380b86bd854cbe73cb7c"}
Sep 11 18:33:29.397854 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:29.397789    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" podStartSLOduration=2.397760512 podStartE2EDuration="2.397760512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:33:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:33:29.39266608 +0000 UTC m=+1072.427882762" watchObservedRunningTime="2026-09-11 18:33:29.397760512 +0000 UTC m=+1072.432977185"
Sep 11 18:33:29.410243 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:29.410215    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d337c0f-905d-4ad4-b58f-52f8f2ca52bd" path="/var/lib/kubelet/pods/4d337c0f-905d-4ad4-b58f-52f8f2ca52bd/volumes"
Sep 11 18:33:29.410578 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:29.410565    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8c22496-8b28-4717-b96c-d77fe4bfc1f8" path="/var/lib/kubelet/pods/e8c22496-8b28-4717-b96c-d77fe4bfc1f8/volumes"
Sep 11 18:33:30.246238 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:30.246158    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:30.251276 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:30.251249    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:30.377518 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:30.377490    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:30.378535 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:30.378514    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:32.391556 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.391528    2578 generic.go:356] "Generic (PLEG): container finished" podID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerID="16786729dfd9f070268ef937e72f2e26a9f2fb9c68a66ebf3ad18e2b3f36c0d4" exitCode=0
Sep 11 18:33:32.392024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.391605    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" event={"ID":"5a3206e7-66ee-41ae-83a6-0c95e53b3e09","Type":"ContainerDied","Data":"16786729dfd9f070268ef937e72f2e26a9f2fb9c68a66ebf3ad18e2b3f36c0d4"}
Sep 11 18:33:32.392024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.391715    2578 scope.go:117] "RemoveContainer" containerID="8c97d0356d247ee8e2557aaddad4f8e2cbeca2e849f176c6e98e2faaa0e1b5fe"
Sep 11 18:33:32.493921 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.493899    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:33:32.542587 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542510    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clthd\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-kube-api-access-clthd\") pod \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") "
Sep 11 18:33:32.542587 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542557    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istiod-ca-cert\") pod \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") "
Sep 11 18:33:32.542819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542590    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-certs\") pod \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") "
Sep 11 18:33:32.542819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542643    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-data\") pod \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") "
Sep 11 18:33:32.542819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542670    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-podinfo\") pod \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") "
Sep 11 18:33:32.542819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542760    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-envoy\") pod \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") "
Sep 11 18:33:32.543038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542804    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-token\") pod \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") "
Sep 11 18:33:32.543038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542860    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-socket\") pod \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") "
Sep 11 18:33:32.543038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542957    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a3206e7-66ee-41ae-83a6-0c95e53b3e09" (UID: "5a3206e7-66ee-41ae-83a6-0c95e53b3e09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:33:32.543038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542978    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a3206e7-66ee-41ae-83a6-0c95e53b3e09" (UID: "5a3206e7-66ee-41ae-83a6-0c95e53b3e09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:33:32.543038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.542964    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-credential-socket\") pod \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\" (UID: \"5a3206e7-66ee-41ae-83a6-0c95e53b3e09\") "
Sep 11 18:33:32.543299 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.543060    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a3206e7-66ee-41ae-83a6-0c95e53b3e09" (UID: "5a3206e7-66ee-41ae-83a6-0c95e53b3e09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:33:32.543299 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.543206    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a3206e7-66ee-41ae-83a6-0c95e53b3e09" (UID: "5a3206e7-66ee-41ae-83a6-0c95e53b3e09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:33:32.543299 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.543290    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a3206e7-66ee-41ae-83a6-0c95e53b3e09" (UID: "5a3206e7-66ee-41ae-83a6-0c95e53b3e09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:33:32.543462 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.543423    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:32.543462 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.543442    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:32.543462 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.543457    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:32.543619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.543469    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:32.543619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.543482    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:32.545078 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.545044    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a3206e7-66ee-41ae-83a6-0c95e53b3e09" (UID: "5a3206e7-66ee-41ae-83a6-0c95e53b3e09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:33:32.545078 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.545063    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a3206e7-66ee-41ae-83a6-0c95e53b3e09" (UID: "5a3206e7-66ee-41ae-83a6-0c95e53b3e09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:33:32.545253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.545236    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-kube-api-access-clthd" (OuterVolumeSpecName: "kube-api-access-clthd") pod "5a3206e7-66ee-41ae-83a6-0c95e53b3e09" (UID: "5a3206e7-66ee-41ae-83a6-0c95e53b3e09"). InnerVolumeSpecName "kube-api-access-clthd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:33:32.545312 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.545259    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a3206e7-66ee-41ae-83a6-0c95e53b3e09" (UID: "5a3206e7-66ee-41ae-83a6-0c95e53b3e09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:33:32.643975 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.643942    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:32.643975 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.643971    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clthd\" (UniqueName: \"kubernetes.io/projected/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-kube-api-access-clthd\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:32.643975 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.643981    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:32.644201 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.643989    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3206e7-66ee-41ae-83a6-0c95e53b3e09-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:33:32.749172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.749139    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"]
Sep 11 18:33:32.749558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.749546    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy"
Sep 11 18:33:32.749602 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.749560    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy"
Sep 11 18:33:32.749647 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.749638    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy"
Sep 11 18:33:32.749713 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.749648    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy"
Sep 11 18:33:32.751999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.751982    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:32.754429 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.754405    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f8e161a6-dockercfg-vd62b\""
Sep 11 18:33:32.754562 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.754408    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f8e161a6\""
Sep 11 18:33:32.763517 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.763493    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"]
Sep 11 18:33:32.846266 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.846176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkwpk\" (UniqueName: \"kubernetes.io/projected/87be8f15-673b-4c1b-a054-9071f308dea6-kube-api-access-qkwpk\") pod \"payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm\" (UID: \"87be8f15-673b-4c1b-a054-9071f308dea6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:32.846422 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.846285    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87be8f15-673b-4c1b-a054-9071f308dea6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm\" (UID: \"87be8f15-673b-4c1b-a054-9071f308dea6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:32.947800 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.947766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkwpk\" (UniqueName: \"kubernetes.io/projected/87be8f15-673b-4c1b-a054-9071f308dea6-kube-api-access-qkwpk\") pod \"payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm\" (UID: \"87be8f15-673b-4c1b-a054-9071f308dea6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:32.947991 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.947835    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87be8f15-673b-4c1b-a054-9071f308dea6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm\" (UID: \"87be8f15-673b-4c1b-a054-9071f308dea6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:32.948484 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.948464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87be8f15-673b-4c1b-a054-9071f308dea6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm\" (UID: \"87be8f15-673b-4c1b-a054-9071f308dea6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:32.956358 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:32.956336    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkwpk\" (UniqueName: \"kubernetes.io/projected/87be8f15-673b-4c1b-a054-9071f308dea6-kube-api-access-qkwpk\") pod \"payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm\" (UID: \"87be8f15-673b-4c1b-a054-9071f308dea6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:33.063007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:33.062978    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:33.334410 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:33.334388    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"]
Sep 11 18:33:33.336530 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:33:33.336502    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87be8f15_673b_4c1b_a054_9071f308dea6.slice/crio-63cf7fcbdbe07fd77c15231ff5fa7b631465556bae9daa3c30b3c362427c772e WatchSource:0}: Error finding container 63cf7fcbdbe07fd77c15231ff5fa7b631465556bae9daa3c30b3c362427c772e: Status 404 returned error can't find the container with id 63cf7fcbdbe07fd77c15231ff5fa7b631465556bae9daa3c30b3c362427c772e
Sep 11 18:33:33.399776 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:33.399750    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"
Sep 11 18:33:33.400171 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:33.399773    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv" event={"ID":"5a3206e7-66ee-41ae-83a6-0c95e53b3e09","Type":"ContainerDied","Data":"88f428c76bad4e5071652aac3fb0645df4a94c813191b6d15ee5cc51f4098920"}
Sep 11 18:33:33.400171 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:33.399844    2578 scope.go:117] "RemoveContainer" containerID="16786729dfd9f070268ef937e72f2e26a9f2fb9c68a66ebf3ad18e2b3f36c0d4"
Sep 11 18:33:33.414454 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:33.414425    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm" event={"ID":"87be8f15-673b-4c1b-a054-9071f308dea6","Type":"ContainerStarted","Data":"63cf7fcbdbe07fd77c15231ff5fa7b631465556bae9daa3c30b3c362427c772e"}
Sep 11 18:33:33.445659 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:33.445627    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"]
Sep 11 18:33:33.465087 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:33.465057    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-414d08da-gw-openshift-default-69b446cb8-86lpv"]
Sep 11 18:33:34.418638 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:34.418598    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm" event={"ID":"87be8f15-673b-4c1b-a054-9071f308dea6","Type":"ContainerStarted","Data":"d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9"}
Sep 11 18:33:34.419043 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:34.418661    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:34.438893 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:34.438843    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm" podStartSLOduration=2.438826655 podStartE2EDuration="2.438826655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:33:34.437523638 +0000 UTC m=+1077.472740313" watchObservedRunningTime="2026-09-11 18:33:34.438826655 +0000 UTC m=+1077.474043339"
Sep 11 18:33:35.410274 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:35.410237    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" path="/var/lib/kubelet/pods/5a3206e7-66ee-41ae-83a6-0c95e53b3e09/volumes"
Sep 11 18:33:44.439263 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.439223    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"]
Sep 11 18:33:44.440740 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.440713    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy"
Sep 11 18:33:44.440899 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.440887    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a3206e7-66ee-41ae-83a6-0c95e53b3e09" containerName="istio-proxy"
Sep 11 18:33:44.445266 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.445243    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.445656 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.445628    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"]
Sep 11 18:33:44.448182 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.448160    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-047453\"/\"openshift-service-ca.crt\""
Sep 11 18:33:44.448319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.448301    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-047453\"/\"default-dockercfg-6srvg\""
Sep 11 18:33:44.449178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.449156    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-047453\"/\"kube-root-ca.crt\""
Sep 11 18:33:44.449488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.449466    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-047453\"/\"simulator-w0-047453-kserve-self-signed-certs\""
Sep 11 18:33:44.564255 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.564222    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tmp-dir\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.564442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.564280    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-dshm\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.564442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.564312    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tls-certs\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.564442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.564348    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-model-cache\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.564442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.564380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5psc\" (UniqueName: \"kubernetes.io/projected/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-kube-api-access-l5psc\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.564442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.564408    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-home\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.665841 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.665810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-dshm\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.665841 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.665844    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tls-certs\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.666076 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.665887    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-model-cache\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.666076 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.665915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5psc\" (UniqueName: \"kubernetes.io/projected/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-kube-api-access-l5psc\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.666076 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.665942    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-home\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.666220 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.666108    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tmp-dir\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.666310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.666289    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-model-cache\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.666353 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.666300    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-home\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.666503 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.666483    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tmp-dir\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.667994 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.667972    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-dshm\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.668273 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.668253    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tls-certs\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.676059 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.676032    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5psc\" (UniqueName: \"kubernetes.io/projected/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-kube-api-access-l5psc\") pod \"simulator-w0-047453-kserve-6df8db4596-l9mtk\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") " pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:44.755310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:44.755216    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:45.081305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:45.081217    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"]
Sep 11 18:33:45.085830 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:33:45.085799    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa39efd2_1cc7_4d1b_b67e_f7a5df7b2423.slice/crio-317174e06423ca6ea10d0e865943ca6c9ec1a0b0892a1a99d7764efa000471ae WatchSource:0}: Error finding container 317174e06423ca6ea10d0e865943ca6c9ec1a0b0892a1a99d7764efa000471ae: Status 404 returned error can't find the container with id 317174e06423ca6ea10d0e865943ca6c9ec1a0b0892a1a99d7764efa000471ae
Sep 11 18:33:45.426638 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:45.426593    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:33:45.481819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:45.481781    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk" event={"ID":"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423","Type":"ContainerStarted","Data":"18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee"}
Sep 11 18:33:45.482327 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:45.481836    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk" event={"ID":"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423","Type":"ContainerStarted","Data":"317174e06423ca6ea10d0e865943ca6c9ec1a0b0892a1a99d7764efa000471ae"}
Sep 11 18:33:45.503284 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:45.503231    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk" podStartSLOduration=1.503214158 podStartE2EDuration="1.503214158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:33:45.502354548 +0000 UTC m=+1088.537571222" watchObservedRunningTime="2026-09-11 18:33:45.503214158 +0000 UTC m=+1088.538430828"
Sep 11 18:33:52.523124 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:52.523090    2578 generic.go:356] "Generic (PLEG): container finished" podID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerID="eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333" exitCode=0
Sep 11 18:33:52.523672 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:52.523163    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" event={"ID":"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1","Type":"ContainerDied","Data":"eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333"}
Sep 11 18:33:52.523672 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:52.523656    2578 scope.go:117] "RemoveContainer" containerID="eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333"
Sep 11 18:33:52.524912 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:52.524882    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:33:52.525013 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:52.524953    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:33:52.525013 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:52.524987    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:33:53.530577 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:53.530543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" event={"ID":"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1","Type":"ContainerStarted","Data":"68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af"}
Sep 11 18:33:54.246561 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:54.246522    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:33:54.246896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:54.246591    2578 patch_prober.go:28] interesting pod/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 18:33:54.246896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:54.246670    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 18:33:54.755997 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:54.755955    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:54.756410 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:54.756225    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:54.768997 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:54.768960    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:55.246331 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:55.246292    2578 patch_prober.go:28] interesting pod/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 18:33:55.246514 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:55.246351    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 18:33:55.554401 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:55.554324    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:33:56.246217 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.246176    2578 patch_prober.go:28] interesting pod/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 18:33:56.246640 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.246238    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 18:33:56.490690 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.490640    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"]
Sep 11 18:33:56.493419 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.493402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"
Sep 11 18:33:56.495940 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.495920    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:33:56.496198 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.496183    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:33:56.504304 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.504283    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"]
Sep 11 18:33:56.591937 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.591898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48cd3499-4185-458c-b962-77402ad4e08c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t\" (UID: \"48cd3499-4185-458c-b962-77402ad4e08c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"
Sep 11 18:33:56.692769 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.692729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48cd3499-4185-458c-b962-77402ad4e08c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t\" (UID: \"48cd3499-4185-458c-b962-77402ad4e08c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"
Sep 11 18:33:56.693338 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.693317    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48cd3499-4185-458c-b962-77402ad4e08c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t\" (UID: \"48cd3499-4185-458c-b962-77402ad4e08c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"
Sep 11 18:33:56.804582 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:56.804499    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"
Sep 11 18:33:57.094806 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:57.094718    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"]
Sep 11 18:33:57.097771 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:33:57.097744    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48cd3499_4185_458c_b962_77402ad4e08c.slice/crio-8e0978744212bc873a7a8f5ecc2410adf515346293b934791923092f9fd4035c WatchSource:0}: Error finding container 8e0978744212bc873a7a8f5ecc2410adf515346293b934791923092f9fd4035c: Status 404 returned error can't find the container with id 8e0978744212bc873a7a8f5ecc2410adf515346293b934791923092f9fd4035c
Sep 11 18:33:57.245902 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:57.245864    2578 patch_prober.go:28] interesting pod/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 18:33:57.246088 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:57.245929    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 18:33:57.554806 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:57.554768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t" event={"ID":"48cd3499-4185-458c-b962-77402ad4e08c","Type":"ContainerStarted","Data":"5baef964c75cc0d2c3b99cbbe334e5452ea412540a35a3e2122b763f199c9b32"}
Sep 11 18:33:57.555257 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:57.554827    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t" event={"ID":"48cd3499-4185-458c-b962-77402ad4e08c","Type":"ContainerStarted","Data":"8e0978744212bc873a7a8f5ecc2410adf515346293b934791923092f9fd4035c"}
Sep 11 18:33:57.573089 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:57.573040    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t" podStartSLOduration=1.5730258049999999 podStartE2EDuration="1.573025805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:33:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:33:57.570466055 +0000 UTC m=+1100.605682731" watchObservedRunningTime="2026-09-11 18:33:57.573025805 +0000 UTC m=+1100.608242474"
Sep 11 18:33:58.246245 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:58.246206    2578 patch_prober.go:28] interesting pod/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 18:33:58.246415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:58.246261    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 18:33:58.561797 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:58.561766    2578 generic.go:356] "Generic (PLEG): container finished" podID="48cd3499-4185-458c-b962-77402ad4e08c" containerID="5baef964c75cc0d2c3b99cbbe334e5452ea412540a35a3e2122b763f199c9b32" exitCode=0
Sep 11 18:33:58.562191 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:58.561828    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t" event={"ID":"48cd3499-4185-458c-b962-77402ad4e08c","Type":"ContainerDied","Data":"5baef964c75cc0d2c3b99cbbe334e5452ea412540a35a3e2122b763f199c9b32"}
Sep 11 18:33:59.245979 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:59.245937    2578 patch_prober.go:28] interesting pod/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 11 18:33:59.246172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:59.246001    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 11 18:33:59.730127 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:59.730098    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"
Sep 11 18:33:59.831274 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:59.831180    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48cd3499-4185-458c-b962-77402ad4e08c-maas-api-service-ca\") pod \"48cd3499-4185-458c-b962-77402ad4e08c\" (UID: \"48cd3499-4185-458c-b962-77402ad4e08c\") "
Sep 11 18:33:59.831500 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:59.831478    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48cd3499-4185-458c-b962-77402ad4e08c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48cd3499-4185-458c-b962-77402ad4e08c" (UID: "48cd3499-4185-458c-b962-77402ad4e08c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:33:59.932431 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:33:59.932397    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48cd3499-4185-458c-b962-77402ad4e08c-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:00.249863 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:00.249827    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:34:00.250244 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:00.250220    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:34:00.250872 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:00.250853    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:34:00.573657 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:00.573570    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"
Sep 11 18:34:00.573657 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:00.573611    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t" event={"ID":"48cd3499-4185-458c-b962-77402ad4e08c","Type":"ContainerDied","Data":"8e0978744212bc873a7a8f5ecc2410adf515346293b934791923092f9fd4035c"}
Sep 11 18:34:00.573900 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:00.573660    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e0978744212bc873a7a8f5ecc2410adf515346293b934791923092f9fd4035c"
Sep 11 18:34:02.551266 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.551229    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"]
Sep 11 18:34:02.551768 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.551467    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm" podUID="87be8f15-673b-4c1b-a054-9071f308dea6" containerName="payload-pre-processing" containerID="cri-o://d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9" gracePeriod=30
Sep 11 18:34:02.747944 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.747886    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"]
Sep 11 18:34:02.752227 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.752153    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f8e161a6-gbj5t"]
Sep 11 18:34:02.815434 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.815362    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:34:02.961867 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.961819    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87be8f15-673b-4c1b-a054-9071f308dea6-plugins-config-volume\") pod \"87be8f15-673b-4c1b-a054-9071f308dea6\" (UID: \"87be8f15-673b-4c1b-a054-9071f308dea6\") "
Sep 11 18:34:02.962030 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.961912    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkwpk\" (UniqueName: \"kubernetes.io/projected/87be8f15-673b-4c1b-a054-9071f308dea6-kube-api-access-qkwpk\") pod \"87be8f15-673b-4c1b-a054-9071f308dea6\" (UID: \"87be8f15-673b-4c1b-a054-9071f308dea6\") "
Sep 11 18:34:02.962268 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.962240    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87be8f15-673b-4c1b-a054-9071f308dea6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "87be8f15-673b-4c1b-a054-9071f308dea6" (UID: "87be8f15-673b-4c1b-a054-9071f308dea6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:34:02.962398 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.962381    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87be8f15-673b-4c1b-a054-9071f308dea6-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:02.964726 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:02.964669    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87be8f15-673b-4c1b-a054-9071f308dea6-kube-api-access-qkwpk" (OuterVolumeSpecName: "kube-api-access-qkwpk") pod "87be8f15-673b-4c1b-a054-9071f308dea6" (UID: "87be8f15-673b-4c1b-a054-9071f308dea6"). InnerVolumeSpecName "kube-api-access-qkwpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:34:03.063969 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.063926    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkwpk\" (UniqueName: \"kubernetes.io/projected/87be8f15-673b-4c1b-a054-9071f308dea6-kube-api-access-qkwpk\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:03.410614 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.410576    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48cd3499-4185-458c-b962-77402ad4e08c" path="/var/lib/kubelet/pods/48cd3499-4185-458c-b962-77402ad4e08c/volumes"
Sep 11 18:34:03.594891 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.594855    2578 generic.go:356] "Generic (PLEG): container finished" podID="87be8f15-673b-4c1b-a054-9071f308dea6" containerID="d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9" exitCode=0
Sep 11 18:34:03.595378 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.594899    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"
Sep 11 18:34:03.595378 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.594943    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm" event={"ID":"87be8f15-673b-4c1b-a054-9071f308dea6","Type":"ContainerDied","Data":"d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9"}
Sep 11 18:34:03.595378 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.595026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm" event={"ID":"87be8f15-673b-4c1b-a054-9071f308dea6","Type":"ContainerDied","Data":"63cf7fcbdbe07fd77c15231ff5fa7b631465556bae9daa3c30b3c362427c772e"}
Sep 11 18:34:03.595378 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.595057    2578 scope.go:117] "RemoveContainer" containerID="d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9"
Sep 11 18:34:03.604275 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.604251    2578 scope.go:117] "RemoveContainer" containerID="d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9"
Sep 11 18:34:03.604586 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:34:03.604566    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9\": container with ID starting with d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9 not found: ID does not exist" containerID="d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9"
Sep 11 18:34:03.604629 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.604594    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9"} err="failed to get container status \"d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9\": rpc error: code = NotFound desc = could not find container \"d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9\": container with ID starting with d12dc75527e5a7e88e9e3a2ce6d828477d037f75d44eb9d28244cb6f4dfa5fa9 not found: ID does not exist"
Sep 11 18:34:03.616086 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.616056    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"]
Sep 11 18:34:03.620746 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.620715    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f8e161a6-75876b77c8-tdzdm"]
Sep 11 18:34:03.851721 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.851622    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"]
Sep 11 18:34:03.851876 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:03.851845    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy" containerID="cri-o://68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af" gracePeriod=30
Sep 11 18:34:04.574654 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.574620    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"]
Sep 11 18:34:04.575105 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.575087    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87be8f15-673b-4c1b-a054-9071f308dea6" containerName="payload-pre-processing"
Sep 11 18:34:04.575253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.575108    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="87be8f15-673b-4c1b-a054-9071f308dea6" containerName="payload-pre-processing"
Sep 11 18:34:04.575253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.575150    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48cd3499-4185-458c-b962-77402ad4e08c" containerName="revoke-keys"
Sep 11 18:34:04.575253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.575159    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="48cd3499-4185-458c-b962-77402ad4e08c" containerName="revoke-keys"
Sep 11 18:34:04.575434 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.575261    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="48cd3499-4185-458c-b962-77402ad4e08c" containerName="revoke-keys"
Sep 11 18:34:04.575434 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.575279    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="87be8f15-673b-4c1b-a054-9071f308dea6" containerName="payload-pre-processing"
Sep 11 18:34:04.578439 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.578418    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.582539 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.582518    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-0b2b4b09-openshift-default-dockercfg-p7gvq\""
Sep 11 18:34:04.593314 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.593283    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"]
Sep 11 18:34:04.684620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.684245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-podinfo\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.684620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.684373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-token\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.684620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.684419    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6cs2\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-kube-api-access-x6cs2\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.684620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.684473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-envoy\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.684620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.684506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istiod-ca-cert\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.684620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.684532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-certs\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.684620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.684598    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-data\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.684620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.684626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-socket\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.685500 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.684655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-credential-socket\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.785491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.785452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-envoy\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.785673 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.785493    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istiod-ca-cert\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.785673 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.785546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-certs\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.785673 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.785600    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-data\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.785673 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.785631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-socket\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.785673 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.785664    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-credential-socket\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.785999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.785958    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-podinfo\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.785999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.785978    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-certs\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.786105 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.786087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-data\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.786160 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.786099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-token\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.786160 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.786104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-credential-socket\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.786261 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.786170    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-socket\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.786261 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.786183    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6cs2\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-kube-api-access-x6cs2\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.786400 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.786375    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istiod-ca-cert\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.787948 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.787928    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-envoy\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.788328 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.788309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-podinfo\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.794732 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.794692    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-token\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.794942 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.794922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6cs2\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-kube-api-access-x6cs2\") pod \"e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") " pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:04.890016 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:04.889915    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:05.175215 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:05.175189    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"]
Sep 11 18:34:05.177487 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:34:05.177458    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8da8c8_e800_43b2_9ef4_11d8a911a0ed.slice/crio-50246c2c13c82f46f87e795af7e9ed1473a7e56bc6dbf8dcf9b5b82ca7c0d061 WatchSource:0}: Error finding container 50246c2c13c82f46f87e795af7e9ed1473a7e56bc6dbf8dcf9b5b82ca7c0d061: Status 404 returned error can't find the container with id 50246c2c13c82f46f87e795af7e9ed1473a7e56bc6dbf8dcf9b5b82ca7c0d061
Sep 11 18:34:05.179774 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:05.179740    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:34:05.179862 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:05.179810    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:34:05.179862 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:05.179840    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:34:05.412592 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:05.412549    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87be8f15-673b-4c1b-a054-9071f308dea6" path="/var/lib/kubelet/pods/87be8f15-673b-4c1b-a054-9071f308dea6/volumes"
Sep 11 18:34:05.610061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:05.609966    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" event={"ID":"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed","Type":"ContainerStarted","Data":"1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55"}
Sep 11 18:34:05.610061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:05.610017    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" event={"ID":"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed","Type":"ContainerStarted","Data":"50246c2c13c82f46f87e795af7e9ed1473a7e56bc6dbf8dcf9b5b82ca7c0d061"}
Sep 11 18:34:05.637230 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:05.637174    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" podStartSLOduration=1.637156095 podStartE2EDuration="1.637156095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:34:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:34:05.635164995 +0000 UTC m=+1108.670381666" watchObservedRunningTime="2026-09-11 18:34:05.637156095 +0000 UTC m=+1108.672372764"
Sep 11 18:34:05.890516 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:05.890414    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:06.895560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:06.895525    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:07.621824 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:07.621791    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:07.622860 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:07.622817    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:09.120180 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.120107    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:34:09.176198 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.176161    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"]
Sep 11 18:34:09.176566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.176552    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy"
Sep 11 18:34:09.176566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.176565    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy"
Sep 11 18:34:09.176763 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.176574    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy"
Sep 11 18:34:09.176763 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.176580    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy"
Sep 11 18:34:09.176763 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.176657    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy"
Sep 11 18:34:09.176763 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.176666    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerName="istio-proxy"
Sep 11 18:34:09.178709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.178688    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:09.182860 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.182837    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-0b2b4b09-dockercfg-z2jnf\""
Sep 11 18:34:09.183445 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.183421    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-0b2b4b09\""
Sep 11 18:34:09.193836 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.193808    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"]
Sep 11 18:34:09.239732 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.239697    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-podinfo\") pod \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") "
Sep 11 18:34:09.240050 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.240026    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-socket\") pod \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") "
Sep 11 18:34:09.240207 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.240191    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istiod-ca-cert\") pod \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") "
Sep 11 18:34:09.240336 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.240323    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-envoy\") pod \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") "
Sep 11 18:34:09.240455 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.240442    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-certs\") pod \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") "
Sep 11 18:34:09.240593 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.240579    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-data\") pod \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") "
Sep 11 18:34:09.240761 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.240745    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-credential-socket\") pod \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") "
Sep 11 18:34:09.240907 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.240894    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-token\") pod \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") "
Sep 11 18:34:09.241006 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.240993    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llw7w\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-kube-api-access-llw7w\") pod \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\" (UID: \"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1\") "
Sep 11 18:34:09.242724 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.242391    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" (UID: "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:09.243654 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.243008    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" (UID: "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:34:09.243654 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.243160    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" (UID: "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:09.243654 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.243286    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" (UID: "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:09.243654 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.243635    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:09.243654 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.243653    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:09.244046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.243666    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:09.244046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.243696    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:09.244144 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.244083    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" (UID: "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:09.251174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.251140    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" (UID: "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:09.262942 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.260018    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" (UID: "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:34:09.262942 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.260824    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-kube-api-access-llw7w" (OuterVolumeSpecName: "kube-api-access-llw7w") pod "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" (UID: "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1"). InnerVolumeSpecName "kube-api-access-llw7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:34:09.262942 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.260885    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" (UID: "5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:34:09.344774 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.344732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7cd8541-c7fc-41ac-b96d-4a37819536ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf\" (UID: \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:09.344954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.344796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skj5w\" (UniqueName: \"kubernetes.io/projected/a7cd8541-c7fc-41ac-b96d-4a37819536ca-kube-api-access-skj5w\") pod \"payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf\" (UID: \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:09.344954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.344858    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:09.344954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.344869    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:09.344954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.344879    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llw7w\" (UniqueName: \"kubernetes.io/projected/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-kube-api-access-llw7w\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:09.344954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.344888    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:09.344954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.344896    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:09.446131 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.446086    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skj5w\" (UniqueName: \"kubernetes.io/projected/a7cd8541-c7fc-41ac-b96d-4a37819536ca-kube-api-access-skj5w\") pod \"payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf\" (UID: \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:09.446338 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.446240    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7cd8541-c7fc-41ac-b96d-4a37819536ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf\" (UID: \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:09.446993 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.446967    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7cd8541-c7fc-41ac-b96d-4a37819536ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf\" (UID: \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:09.455370 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.455343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skj5w\" (UniqueName: \"kubernetes.io/projected/a7cd8541-c7fc-41ac-b96d-4a37819536ca-kube-api-access-skj5w\") pod \"payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf\" (UID: \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:09.490232 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.489875    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:09.634436 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.634400    2578 generic.go:356] "Generic (PLEG): container finished" podID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" containerID="68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af" exitCode=0
Sep 11 18:34:09.634626 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.634448    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"
Sep 11 18:34:09.634626 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.634486    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" event={"ID":"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1","Type":"ContainerDied","Data":"68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af"}
Sep 11 18:34:09.634626 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.634537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt" event={"ID":"5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1","Type":"ContainerDied","Data":"cf83264668c6c0f5e4c78a745f76b813802b0ea633e0380b86bd854cbe73cb7c"}
Sep 11 18:34:09.634626 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.634565    2578 scope.go:117] "RemoveContainer" containerID="68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af"
Sep 11 18:34:09.643897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.643878    2578 scope.go:117] "RemoveContainer" containerID="eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333"
Sep 11 18:34:09.652404 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.652379    2578 scope.go:117] "RemoveContainer" containerID="68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af"
Sep 11 18:34:09.652856 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:34:09.652739    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af\": container with ID starting with 68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af not found: ID does not exist" containerID="68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af"
Sep 11 18:34:09.652856 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.652778    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af"} err="failed to get container status \"68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af\": rpc error: code = NotFound desc = could not find container \"68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af\": container with ID starting with 68b983c02b7ea66e24a05ed97b84285a81863af828cf67ce2df5c0161228b2af not found: ID does not exist"
Sep 11 18:34:09.652856 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.652803    2578 scope.go:117] "RemoveContainer" containerID="eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333"
Sep 11 18:34:09.653128 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:34:09.653110    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333\": container with ID starting with eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333 not found: ID does not exist" containerID="eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333"
Sep 11 18:34:09.653128 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.653135    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333"} err="failed to get container status \"eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333\": rpc error: code = NotFound desc = could not find container \"eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333\": container with ID starting with eb102700e11a295140c5350a45102f5330ef52de225976dec9d4cbe4ababe333 not found: ID does not exist"
Sep 11 18:34:09.656453 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.656431    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"]
Sep 11 18:34:09.658395 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.658374    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f8e161a6-openshift-default-7b6b9745d4-5ntwt"]
Sep 11 18:34:09.775602 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:09.775515    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"]
Sep 11 18:34:09.779015 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:34:09.778949    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7cd8541_c7fc_41ac_b96d_4a37819536ca.slice/crio-635a83a9cd219e7a2796dc0aa25bce1c348ff8316eb56345149dab80685ae821 WatchSource:0}: Error finding container 635a83a9cd219e7a2796dc0aa25bce1c348ff8316eb56345149dab80685ae821: Status 404 returned error can't find the container with id 635a83a9cd219e7a2796dc0aa25bce1c348ff8316eb56345149dab80685ae821
Sep 11 18:34:10.641659 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:10.641616    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf" event={"ID":"a7cd8541-c7fc-41ac-b96d-4a37819536ca","Type":"ContainerStarted","Data":"2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168"}
Sep 11 18:34:10.642175 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:10.641704    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf" event={"ID":"a7cd8541-c7fc-41ac-b96d-4a37819536ca","Type":"ContainerStarted","Data":"635a83a9cd219e7a2796dc0aa25bce1c348ff8316eb56345149dab80685ae821"}
Sep 11 18:34:10.642175 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:10.641746    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:10.663100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:10.663028    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf" podStartSLOduration=1.663006557 podStartE2EDuration="1.663006557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:34:10.662317331 +0000 UTC m=+1113.697534015" watchObservedRunningTime="2026-09-11 18:34:10.663006557 +0000 UTC m=+1113.698223230"
Sep 11 18:34:11.413964 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:11.413925    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1" path="/var/lib/kubelet/pods/5c723f6f-e462-4e0b-b53d-f2e5ec5ae6d1/volumes"
Sep 11 18:34:21.652134 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:21.652102    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:31.770249 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.770172    2578 generic.go:356] "Generic (PLEG): container finished" podID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerID="1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55" exitCode=0
Sep 11 18:34:31.770640 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.770248    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" event={"ID":"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed","Type":"ContainerDied","Data":"1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55"}
Sep 11 18:34:31.770710 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.770661    2578 scope.go:117] "RemoveContainer" containerID="1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55"
Sep 11 18:34:31.771934 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.771905    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:34:31.772039 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.771977    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:34:31.772039 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.772008    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:34:31.871178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.871145    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"]
Sep 11 18:34:31.874051 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.874011    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"
Sep 11 18:34:31.877347 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.877325    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:34:31.877600 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.877579    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:34:31.885313 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.885281    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"]
Sep 11 18:34:31.977338 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:31.977300    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9646924-eb81-4fd3-ab41-b1a159a40dc8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b\" (UID: \"e9646924-eb81-4fd3-ab41-b1a159a40dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"
Sep 11 18:34:32.078308 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:32.078272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9646924-eb81-4fd3-ab41-b1a159a40dc8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b\" (UID: \"e9646924-eb81-4fd3-ab41-b1a159a40dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"
Sep 11 18:34:32.078927 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:32.078908    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9646924-eb81-4fd3-ab41-b1a159a40dc8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b\" (UID: \"e9646924-eb81-4fd3-ab41-b1a159a40dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"
Sep 11 18:34:32.186957 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:32.186926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"
Sep 11 18:34:32.497317 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:32.497186    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"]
Sep 11 18:34:32.499656 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:34:32.499627    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9646924_eb81_4fd3_ab41_b1a159a40dc8.slice/crio-f2e4a4f58cb1391e86b18202983966ecd1250b8fbc75d89be6e0794b04dcf3af WatchSource:0}: Error finding container f2e4a4f58cb1391e86b18202983966ecd1250b8fbc75d89be6e0794b04dcf3af: Status 404 returned error can't find the container with id f2e4a4f58cb1391e86b18202983966ecd1250b8fbc75d89be6e0794b04dcf3af
Sep 11 18:34:32.778331 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:32.778246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b" event={"ID":"e9646924-eb81-4fd3-ab41-b1a159a40dc8","Type":"ContainerStarted","Data":"f2e4a4f58cb1391e86b18202983966ecd1250b8fbc75d89be6e0794b04dcf3af"}
Sep 11 18:34:32.780247 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:32.780216    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" event={"ID":"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed","Type":"ContainerStarted","Data":"da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8"}
Sep 11 18:34:32.890581 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:32.890549    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:32.891726 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:32.891673    2578 patch_prober.go:28] interesting pod/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 11 18:34:32.891847 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:32.891750    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 11 18:34:33.787152 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:33.787058    2578 generic.go:356] "Generic (PLEG): container finished" podID="e9646924-eb81-4fd3-ab41-b1a159a40dc8" containerID="dd862c909b8c36d0ef132d0681e59467c5d3e1800d680edc1b1d7330a3fd2c0e" exitCode=0
Sep 11 18:34:33.787538 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:33.787141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b" event={"ID":"e9646924-eb81-4fd3-ab41-b1a159a40dc8","Type":"ContainerDied","Data":"dd862c909b8c36d0ef132d0681e59467c5d3e1800d680edc1b1d7330a3fd2c0e"}
Sep 11 18:34:33.891104 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:33.891068    2578 patch_prober.go:28] interesting pod/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 11 18:34:33.891295 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:33.891125    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 11 18:34:34.891072 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:34.891039    2578 patch_prober.go:28] interesting pod/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 11 18:34:34.891546 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:34.891094    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 11 18:34:34.985362 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:34.985339    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"
Sep 11 18:34:35.126757 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:35.124153    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9646924-eb81-4fd3-ab41-b1a159a40dc8-maas-api-service-ca\") pod \"e9646924-eb81-4fd3-ab41-b1a159a40dc8\" (UID: \"e9646924-eb81-4fd3-ab41-b1a159a40dc8\") "
Sep 11 18:34:35.126757 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:35.124863    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9646924-eb81-4fd3-ab41-b1a159a40dc8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9646924-eb81-4fd3-ab41-b1a159a40dc8" (UID: "e9646924-eb81-4fd3-ab41-b1a159a40dc8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:34:35.225507 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:35.225461    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9646924-eb81-4fd3-ab41-b1a159a40dc8-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:35.799982 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:35.799898    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"
Sep 11 18:34:35.799982 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:35.799931    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b" event={"ID":"e9646924-eb81-4fd3-ab41-b1a159a40dc8","Type":"ContainerDied","Data":"f2e4a4f58cb1391e86b18202983966ecd1250b8fbc75d89be6e0794b04dcf3af"}
Sep 11 18:34:35.800176 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:35.799985    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2e4a4f58cb1391e86b18202983966ecd1250b8fbc75d89be6e0794b04dcf3af"
Sep 11 18:34:35.890908 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:35.890871    2578 patch_prober.go:28] interesting pod/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 11 18:34:35.891161 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:35.890933    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 11 18:34:36.890289 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:36.890247    2578 patch_prober.go:28] interesting pod/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 11 18:34:36.890473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:36.890319    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 11 18:34:37.622412 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:37.622373    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:37.896810 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:37.896722    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:37.897897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:37.897871    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:39.307796 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.307765    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"]
Sep 11 18:34:39.308184 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.308169    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9646924-eb81-4fd3-ab41-b1a159a40dc8" containerName="revoke-keys"
Sep 11 18:34:39.308184 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.308185    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9646924-eb81-4fd3-ab41-b1a159a40dc8" containerName="revoke-keys"
Sep 11 18:34:39.308271 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.308263    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9646924-eb81-4fd3-ab41-b1a159a40dc8" containerName="revoke-keys"
Sep 11 18:34:39.311252 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.311232    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.314939 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.314898    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4d5a75\"/\"test-model-4d5a75-kserve-self-signed-certs\""
Sep 11 18:34:39.315946 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.315924    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4d5a75\"/\"default-dockercfg-8q27z\""
Sep 11 18:34:39.316070 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.315972    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4d5a75\"/\"kube-root-ca.crt\""
Sep 11 18:34:39.316070 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.315973    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4d5a75\"/\"openshift-service-ca.crt\""
Sep 11 18:34:39.325651 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.325629    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"]
Sep 11 18:34:39.369300 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.369265    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"]
Sep 11 18:34:39.369773 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.369744    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf" podUID="a7cd8541-c7fc-41ac-b96d-4a37819536ca" containerName="payload-pre-processing" containerID="cri-o://2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168" gracePeriod=30
Sep 11 18:34:39.477140 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.477104    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-home\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.477322 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.477162    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jf8z\" (UniqueName: \"kubernetes.io/projected/02230d2f-ccad-496b-8e95-387d1cef2548-kube-api-access-6jf8z\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.477322 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.477239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-tmp-dir\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.477322 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.477296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-dshm\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.477492 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.477416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02230d2f-ccad-496b-8e95-387d1cef2548-tls-certs\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.477492 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.477457    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-model-cache\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.553116 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.553083    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"]
Sep 11 18:34:39.556506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.556478    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0b2b4b09-g2k5b"]
Sep 11 18:34:39.578289 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.578259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02230d2f-ccad-496b-8e95-387d1cef2548-tls-certs\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.578453 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.578307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-model-cache\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.578453 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.578441    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-home\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.578588 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.578526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jf8z\" (UniqueName: \"kubernetes.io/projected/02230d2f-ccad-496b-8e95-387d1cef2548-kube-api-access-6jf8z\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.578588 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.578576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-tmp-dir\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.578724 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.578654    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-dshm\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.578773 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.578723    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-model-cache\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.579037 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.578993    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-home\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.579202 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.579156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-tmp-dir\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.580942 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.580922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-dshm\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.581430 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.581407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02230d2f-ccad-496b-8e95-387d1cef2548-tls-certs\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.587075 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.587055    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jf8z\" (UniqueName: \"kubernetes.io/projected/02230d2f-ccad-496b-8e95-387d1cef2548-kube-api-access-6jf8z\") pod \"test-model-4d5a75-kserve-8577cbcc69-bchb6\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") " pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.621222 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.621166    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:39.699813 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.699559    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"]
Sep 11 18:34:39.706756 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.704748    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:39.832420 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.832326    2578 generic.go:356] "Generic (PLEG): container finished" podID="a7cd8541-c7fc-41ac-b96d-4a37819536ca" containerID="2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168" exitCode=0
Sep 11 18:34:39.832420 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.832377    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"
Sep 11 18:34:39.832658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.832367    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf" event={"ID":"a7cd8541-c7fc-41ac-b96d-4a37819536ca","Type":"ContainerDied","Data":"2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168"}
Sep 11 18:34:39.832658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.832515    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf" event={"ID":"a7cd8541-c7fc-41ac-b96d-4a37819536ca","Type":"ContainerDied","Data":"635a83a9cd219e7a2796dc0aa25bce1c348ff8316eb56345149dab80685ae821"}
Sep 11 18:34:39.832658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.832544    2578 scope.go:117] "RemoveContainer" containerID="2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168"
Sep 11 18:34:39.832658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.832625    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy" containerID="cri-o://da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8" gracePeriod=30
Sep 11 18:34:39.842721 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.842698    2578 scope.go:117] "RemoveContainer" containerID="2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168"
Sep 11 18:34:39.843160 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:34:39.843136    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168\": container with ID starting with 2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168 not found: ID does not exist" containerID="2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168"
Sep 11 18:34:39.843239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.843173    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168"} err="failed to get container status \"2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168\": rpc error: code = NotFound desc = could not find container \"2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168\": container with ID starting with 2da4a7f1f57dbfd3553750fdb81ab42d0ec5437d519d1d0f40d80321d185f168 not found: ID does not exist"
Sep 11 18:34:39.890658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.890621    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skj5w\" (UniqueName: \"kubernetes.io/projected/a7cd8541-c7fc-41ac-b96d-4a37819536ca-kube-api-access-skj5w\") pod \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\" (UID: \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\") "
Sep 11 18:34:39.890851 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.890670    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7cd8541-c7fc-41ac-b96d-4a37819536ca-plugins-config-volume\") pod \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\" (UID: \"a7cd8541-c7fc-41ac-b96d-4a37819536ca\") "
Sep 11 18:34:39.891132 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.891105    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7cd8541-c7fc-41ac-b96d-4a37819536ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7cd8541-c7fc-41ac-b96d-4a37819536ca" (UID: "a7cd8541-c7fc-41ac-b96d-4a37819536ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:34:39.892905 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.892885    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7cd8541-c7fc-41ac-b96d-4a37819536ca-kube-api-access-skj5w" (OuterVolumeSpecName: "kube-api-access-skj5w") pod "a7cd8541-c7fc-41ac-b96d-4a37819536ca" (UID: "a7cd8541-c7fc-41ac-b96d-4a37819536ca"). InnerVolumeSpecName "kube-api-access-skj5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:34:39.959973 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.959942    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"]
Sep 11 18:34:39.961811 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:34:39.961779    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02230d2f_ccad_496b_8e95_387d1cef2548.slice/crio-1d2812e9fcb92b8bef9968a2f1a324d3e3f67c4b932806cd74f7011af35fb2b1 WatchSource:0}: Error finding container 1d2812e9fcb92b8bef9968a2f1a324d3e3f67c4b932806cd74f7011af35fb2b1: Status 404 returned error can't find the container with id 1d2812e9fcb92b8bef9968a2f1a324d3e3f67c4b932806cd74f7011af35fb2b1
Sep 11 18:34:39.991847 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.991822    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skj5w\" (UniqueName: \"kubernetes.io/projected/a7cd8541-c7fc-41ac-b96d-4a37819536ca-kube-api-access-skj5w\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:39.991847 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:39.991847    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7cd8541-c7fc-41ac-b96d-4a37819536ca-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:40.156421 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:40.156394    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"]
Sep 11 18:34:40.166006 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:40.165978    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0b2b4b09-67456446cd-wpltf"]
Sep 11 18:34:40.844182 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:40.844147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6" event={"ID":"02230d2f-ccad-496b-8e95-387d1cef2548","Type":"ContainerStarted","Data":"ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e"}
Sep 11 18:34:40.844182 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:40.844189    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6" event={"ID":"02230d2f-ccad-496b-8e95-387d1cef2548","Type":"ContainerStarted","Data":"1d2812e9fcb92b8bef9968a2f1a324d3e3f67c4b932806cd74f7011af35fb2b1"}
Sep 11 18:34:40.865080 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:40.865028    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6" podStartSLOduration=1.865011585 podStartE2EDuration="1.865011585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:34:40.863550236 +0000 UTC m=+1143.898766945" watchObservedRunningTime="2026-09-11 18:34:40.865011585 +0000 UTC m=+1143.900228255"
Sep 11 18:34:41.410688 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:41.410645    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7cd8541-c7fc-41ac-b96d-4a37819536ca" path="/var/lib/kubelet/pods/a7cd8541-c7fc-41ac-b96d-4a37819536ca/volumes"
Sep 11 18:34:41.411032 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:41.411018    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9646924-eb81-4fd3-ab41-b1a159a40dc8" path="/var/lib/kubelet/pods/e9646924-eb81-4fd3-ab41-b1a159a40dc8/volumes"
Sep 11 18:34:45.085944 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.085916    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:45.136840 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.136811    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istiod-ca-cert\") pod \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") "
Sep 11 18:34:45.137027 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.136857    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-certs\") pod \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") "
Sep 11 18:34:45.137027 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.136888    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-socket\") pod \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") "
Sep 11 18:34:45.137027 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.136957    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-envoy\") pod \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") "
Sep 11 18:34:45.137027 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.136987    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-podinfo\") pod \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") "
Sep 11 18:34:45.137027 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137022    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-credential-socket\") pod \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") "
Sep 11 18:34:45.137303 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137047    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-data\") pod \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") "
Sep 11 18:34:45.137303 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137073    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6cs2\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-kube-api-access-x6cs2\") pod \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") "
Sep 11 18:34:45.137303 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137117    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-token\") pod \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\" (UID: \"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed\") "
Sep 11 18:34:45.137303 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137207    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" (UID: "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:45.137303 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137262    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" (UID: "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:45.137566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137315    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" (UID: "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:34:45.137566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137361    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" (UID: "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:45.137566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137460    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" (UID: "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:45.137764 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137603    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:45.137764 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137621    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:45.137764 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137636    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:45.137764 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137650    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:45.137764 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.137663    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:45.139519 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.139489    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" (UID: "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:45.139890 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.139863    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" (UID: "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:34:45.139984 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.139904    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-kube-api-access-x6cs2" (OuterVolumeSpecName: "kube-api-access-x6cs2") pod "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" (UID: "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed"). InnerVolumeSpecName "kube-api-access-x6cs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:34:45.139984 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.139919    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" (UID: "fb8da8c8-e800-43b2-9ef4-11d8a911a0ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:34:45.239064 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.239032    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:45.239064 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.239058    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6cs2\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-kube-api-access-x6cs2\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:45.239064 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.239069    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:45.239283 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.239077    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:45.873919 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.873840    2578 generic.go:356] "Generic (PLEG): container finished" podID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerID="da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8" exitCode=0
Sep 11 18:34:45.873919 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.873891    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"
Sep 11 18:34:45.874100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.873925    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" event={"ID":"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed","Type":"ContainerDied","Data":"da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8"}
Sep 11 18:34:45.874100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.873987    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm" event={"ID":"fb8da8c8-e800-43b2-9ef4-11d8a911a0ed","Type":"ContainerDied","Data":"50246c2c13c82f46f87e795af7e9ed1473a7e56bc6dbf8dcf9b5b82ca7c0d061"}
Sep 11 18:34:45.874100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.874012    2578 scope.go:117] "RemoveContainer" containerID="da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8"
Sep 11 18:34:45.883056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.883038    2578 scope.go:117] "RemoveContainer" containerID="1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55"
Sep 11 18:34:45.891191 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.891176    2578 scope.go:117] "RemoveContainer" containerID="da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8"
Sep 11 18:34:45.891456 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:34:45.891435    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8\": container with ID starting with da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8 not found: ID does not exist" containerID="da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8"
Sep 11 18:34:45.891550 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.891461    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8"} err="failed to get container status \"da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8\": rpc error: code = NotFound desc = could not find container \"da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8\": container with ID starting with da8cbc4556b991c708c16b73ec3c26891dc30e1dad7bb492a6fe051e7aaf34a8 not found: ID does not exist"
Sep 11 18:34:45.891550 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.891479    2578 scope.go:117] "RemoveContainer" containerID="1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55"
Sep 11 18:34:45.891704 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:34:45.891688    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55\": container with ID starting with 1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55 not found: ID does not exist" containerID="1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55"
Sep 11 18:34:45.891744 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.891707    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55"} err="failed to get container status \"1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55\": rpc error: code = NotFound desc = could not find container \"1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55\": container with ID starting with 1a0562f0f608499025091dd63db5e777f46215d1f803d037c870e53acf623a55 not found: ID does not exist"
Sep 11 18:34:45.895497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.895474    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"]
Sep 11 18:34:45.901954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:45.901936    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-0b2b4b09-openshift-default-694d65944-4szjm"]
Sep 11 18:34:47.410223 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.410191    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" path="/var/lib/kubelet/pods/fb8da8c8-e800-43b2-9ef4-11d8a911a0ed/volumes"
Sep 11 18:34:47.434548 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.434521    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"]
Sep 11 18:34:47.434925 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.434911    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy"
Sep 11 18:34:47.434976 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.434926    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy"
Sep 11 18:34:47.434976 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.434945    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7cd8541-c7fc-41ac-b96d-4a37819536ca" containerName="payload-pre-processing"
Sep 11 18:34:47.434976 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.434950    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7cd8541-c7fc-41ac-b96d-4a37819536ca" containerName="payload-pre-processing"
Sep 11 18:34:47.435105 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.435013    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7cd8541-c7fc-41ac-b96d-4a37819536ca" containerName="payload-pre-processing"
Sep 11 18:34:47.435105 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.435024    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy"
Sep 11 18:34:47.435105 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.435032    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy"
Sep 11 18:34:47.438062 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.438046    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"
Sep 11 18:34:47.440595 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.440560    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:34:47.440782 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.440766    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:34:47.447666 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.447637    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"]
Sep 11 18:34:47.463278 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.463252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22654a6d-a804-45a0-9cb7-bcfea6bc2448-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-536a4d-gfp2p\" (UID: \"22654a6d-a804-45a0-9cb7-bcfea6bc2448\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"
Sep 11 18:34:47.564506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.564464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22654a6d-a804-45a0-9cb7-bcfea6bc2448-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-536a4d-gfp2p\" (UID: \"22654a6d-a804-45a0-9cb7-bcfea6bc2448\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"
Sep 11 18:34:47.565120 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.565099    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22654a6d-a804-45a0-9cb7-bcfea6bc2448-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-536a4d-gfp2p\" (UID: \"22654a6d-a804-45a0-9cb7-bcfea6bc2448\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"
Sep 11 18:34:47.749162 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:47.749074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"
Sep 11 18:34:48.022765 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:48.022736    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"]
Sep 11 18:34:48.024863 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:34:48.024835    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22654a6d_a804_45a0_9cb7_bcfea6bc2448.slice/crio-0cc4adc83ac615d9566864e5f1d203f2d5d9aaa5ce3a4d538f34406072d9daf0 WatchSource:0}: Error finding container 0cc4adc83ac615d9566864e5f1d203f2d5d9aaa5ce3a4d538f34406072d9daf0: Status 404 returned error can't find the container with id 0cc4adc83ac615d9566864e5f1d203f2d5d9aaa5ce3a4d538f34406072d9daf0
Sep 11 18:34:48.894065 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:48.894030    2578 generic.go:356] "Generic (PLEG): container finished" podID="22654a6d-a804-45a0-9cb7-bcfea6bc2448" containerID="6ae48d456edac0a10b13bbfc9bc1b59e270a98c02124ba3aaaae6e9a263bf9a4" exitCode=0
Sep 11 18:34:48.894499 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:48.894112    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p" event={"ID":"22654a6d-a804-45a0-9cb7-bcfea6bc2448","Type":"ContainerDied","Data":"6ae48d456edac0a10b13bbfc9bc1b59e270a98c02124ba3aaaae6e9a263bf9a4"}
Sep 11 18:34:48.894499 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:48.894163    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p" event={"ID":"22654a6d-a804-45a0-9cb7-bcfea6bc2448","Type":"ContainerStarted","Data":"0cc4adc83ac615d9566864e5f1d203f2d5d9aaa5ce3a4d538f34406072d9daf0"}
Sep 11 18:34:49.621417 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:49.621384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:49.621576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:49.621434    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:49.634245 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:49.634217    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:49.911239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:49.911208    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:34:50.038899 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:50.038875    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"
Sep 11 18:34:50.088980 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:50.088944    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22654a6d-a804-45a0-9cb7-bcfea6bc2448-maas-api-service-ca\") pod \"22654a6d-a804-45a0-9cb7-bcfea6bc2448\" (UID: \"22654a6d-a804-45a0-9cb7-bcfea6bc2448\") "
Sep 11 18:34:50.089263 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:50.089241    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22654a6d-a804-45a0-9cb7-bcfea6bc2448-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "22654a6d-a804-45a0-9cb7-bcfea6bc2448" (UID: "22654a6d-a804-45a0-9cb7-bcfea6bc2448"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:34:50.089331 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:50.089321    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22654a6d-a804-45a0-9cb7-bcfea6bc2448-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:50.905729 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:50.905701    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"
Sep 11 18:34:50.905900 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:50.905723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p" event={"ID":"22654a6d-a804-45a0-9cb7-bcfea6bc2448","Type":"ContainerDied","Data":"0cc4adc83ac615d9566864e5f1d203f2d5d9aaa5ce3a4d538f34406072d9daf0"}
Sep 11 18:34:50.905900 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:50.905777    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cc4adc83ac615d9566864e5f1d203f2d5d9aaa5ce3a4d538f34406072d9daf0"
Sep 11 18:34:58.067560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.067528    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"]
Sep 11 18:34:58.068083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.067758    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv" podUID="fca43c26-4def-43f1-8783-31474348edd9" containerName="payload-processing" containerID="cri-o://dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9" gracePeriod=30
Sep 11 18:34:58.346954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.346923    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:34:58.448482 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.448438    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"]
Sep 11 18:34:58.450456 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.450435    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-536a4d-gfp2p"]
Sep 11 18:34:58.478996 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.478961    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9b9r\" (UniqueName: \"kubernetes.io/projected/fca43c26-4def-43f1-8783-31474348edd9-kube-api-access-f9b9r\") pod \"fca43c26-4def-43f1-8783-31474348edd9\" (UID: \"fca43c26-4def-43f1-8783-31474348edd9\") "
Sep 11 18:34:58.479143 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.479056    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca43c26-4def-43f1-8783-31474348edd9-plugins-config-volume\") pod \"fca43c26-4def-43f1-8783-31474348edd9\" (UID: \"fca43c26-4def-43f1-8783-31474348edd9\") "
Sep 11 18:34:58.479436 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.479405    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fca43c26-4def-43f1-8783-31474348edd9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fca43c26-4def-43f1-8783-31474348edd9" (UID: "fca43c26-4def-43f1-8783-31474348edd9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:34:58.481185 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.481161    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca43c26-4def-43f1-8783-31474348edd9-kube-api-access-f9b9r" (OuterVolumeSpecName: "kube-api-access-f9b9r") pod "fca43c26-4def-43f1-8783-31474348edd9" (UID: "fca43c26-4def-43f1-8783-31474348edd9"). InnerVolumeSpecName "kube-api-access-f9b9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:34:58.580371 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.580274    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca43c26-4def-43f1-8783-31474348edd9-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:58.580371 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.580307    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9b9r\" (UniqueName: \"kubernetes.io/projected/fca43c26-4def-43f1-8783-31474348edd9-kube-api-access-f9b9r\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:34:58.961370 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.961335    2578 generic.go:356] "Generic (PLEG): container finished" podID="fca43c26-4def-43f1-8783-31474348edd9" containerID="dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9" exitCode=0
Sep 11 18:34:58.961540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.961386    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"
Sep 11 18:34:58.961540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.961418    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv" event={"ID":"fca43c26-4def-43f1-8783-31474348edd9","Type":"ContainerDied","Data":"dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9"}
Sep 11 18:34:58.961540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.961490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv" event={"ID":"fca43c26-4def-43f1-8783-31474348edd9","Type":"ContainerDied","Data":"abefa7d9abab5f13864137bf3314ddddf5bb0c745e7ce362d37a43a531bdda66"}
Sep 11 18:34:58.961540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.961512    2578 scope.go:117] "RemoveContainer" containerID="dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9"
Sep 11 18:34:58.971113 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.971096    2578 scope.go:117] "RemoveContainer" containerID="dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9"
Sep 11 18:34:58.971406 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:34:58.971388    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9\": container with ID starting with dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9 not found: ID does not exist" containerID="dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9"
Sep 11 18:34:58.971452 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.971414    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9"} err="failed to get container status \"dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9\": rpc error: code = NotFound desc = could not find container \"dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9\": container with ID starting with dbf9b4e4c446cb18154e6931746567f578d1beabffaaeca5dcdccb84647b92d9 not found: ID does not exist"
Sep 11 18:34:58.985857 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.985836    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"]
Sep 11 18:34:58.987929 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:58.987909    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-536a4d-7559ff877f-gcngv"]
Sep 11 18:34:59.409799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:59.409770    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22654a6d-a804-45a0-9cb7-bcfea6bc2448" path="/var/lib/kubelet/pods/22654a6d-a804-45a0-9cb7-bcfea6bc2448/volumes"
Sep 11 18:34:59.410407 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:34:59.410331    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fca43c26-4def-43f1-8783-31474348edd9" path="/var/lib/kubelet/pods/fca43c26-4def-43f1-8783-31474348edd9/volumes"
Sep 11 18:35:37.448321 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:37.448292    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:35:37.452407 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:37.452381    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:35:37.453528 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:37.453509    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:35:37.457796 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:37.457777    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:35:51.422739 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.422706    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"]
Sep 11 18:35:51.423189 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.423104    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca43c26-4def-43f1-8783-31474348edd9" containerName="payload-processing"
Sep 11 18:35:51.423189 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.423118    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca43c26-4def-43f1-8783-31474348edd9" containerName="payload-processing"
Sep 11 18:35:51.423189 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.423131    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22654a6d-a804-45a0-9cb7-bcfea6bc2448" containerName="revoke-keys"
Sep 11 18:35:51.423189 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.423141    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="22654a6d-a804-45a0-9cb7-bcfea6bc2448" containerName="revoke-keys"
Sep 11 18:35:51.423189 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.423152    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy"
Sep 11 18:35:51.423189 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.423161    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb8da8c8-e800-43b2-9ef4-11d8a911a0ed" containerName="istio-proxy"
Sep 11 18:35:51.423520 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.423225    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca43c26-4def-43f1-8783-31474348edd9" containerName="payload-processing"
Sep 11 18:35:51.423520 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.423240    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="22654a6d-a804-45a0-9cb7-bcfea6bc2448" containerName="revoke-keys"
Sep 11 18:35:51.426524 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.426504    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:35:51.429873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.429852    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-a228c4-dockercfg-frczv\""
Sep 11 18:35:51.430110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.430095    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-a228c4\""
Sep 11 18:35:51.436128 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.436105    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"]
Sep 11 18:35:51.438594 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.438576    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:35:51.457874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.457804    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"]
Sep 11 18:35:51.460991 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.460965    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"]
Sep 11 18:35:51.513432 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.513359    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29rwh\" (UniqueName: \"kubernetes.io/projected/d93b6c95-1f08-438c-9979-9363303e8818-kube-api-access-29rwh\") pod \"payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj\" (UID: \"d93b6c95-1f08-438c-9979-9363303e8818\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:35:51.513619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.513491    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d93b6c95-1f08-438c-9979-9363303e8818-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj\" (UID: \"d93b6c95-1f08-438c-9979-9363303e8818\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:35:51.513619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.513554    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bd790aa-d848-4a74-aded-92c7d67d122d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc\" (UID: \"5bd790aa-d848-4a74-aded-92c7d67d122d\") " pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:35:51.513619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.513582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x24gh\" (UniqueName: \"kubernetes.io/projected/5bd790aa-d848-4a74-aded-92c7d67d122d-kube-api-access-x24gh\") pod \"payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc\" (UID: \"5bd790aa-d848-4a74-aded-92c7d67d122d\") " pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:35:51.614857 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.614822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d93b6c95-1f08-438c-9979-9363303e8818-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj\" (UID: \"d93b6c95-1f08-438c-9979-9363303e8818\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:35:51.615042 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.614881    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bd790aa-d848-4a74-aded-92c7d67d122d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc\" (UID: \"5bd790aa-d848-4a74-aded-92c7d67d122d\") " pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:35:51.615042 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.614901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x24gh\" (UniqueName: \"kubernetes.io/projected/5bd790aa-d848-4a74-aded-92c7d67d122d-kube-api-access-x24gh\") pod \"payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc\" (UID: \"5bd790aa-d848-4a74-aded-92c7d67d122d\") " pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:35:51.615042 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.614933    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29rwh\" (UniqueName: \"kubernetes.io/projected/d93b6c95-1f08-438c-9979-9363303e8818-kube-api-access-29rwh\") pod \"payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj\" (UID: \"d93b6c95-1f08-438c-9979-9363303e8818\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:35:51.615616 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.615596    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d93b6c95-1f08-438c-9979-9363303e8818-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj\" (UID: \"d93b6c95-1f08-438c-9979-9363303e8818\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:35:51.615657 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.615594    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bd790aa-d848-4a74-aded-92c7d67d122d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc\" (UID: \"5bd790aa-d848-4a74-aded-92c7d67d122d\") " pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:35:51.624247 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.624217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29rwh\" (UniqueName: \"kubernetes.io/projected/d93b6c95-1f08-438c-9979-9363303e8818-kube-api-access-29rwh\") pod \"payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj\" (UID: \"d93b6c95-1f08-438c-9979-9363303e8818\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:35:51.624801 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.624783    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x24gh\" (UniqueName: \"kubernetes.io/projected/5bd790aa-d848-4a74-aded-92c7d67d122d-kube-api-access-x24gh\") pod \"payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc\" (UID: \"5bd790aa-d848-4a74-aded-92c7d67d122d\") " pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:35:51.738891 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.738808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:35:51.749205 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:51.749176    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:35:52.033972 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.033888    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"]
Sep 11 18:35:52.041082 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:35:52.041053    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd93b6c95_1f08_438c_9979_9363303e8818.slice/crio-ef62dc6212256618ea4acf3fe644979b560a36ede888abfd9cd3bf529b552dce WatchSource:0}: Error finding container ef62dc6212256618ea4acf3fe644979b560a36ede888abfd9cd3bf529b552dce: Status 404 returned error can't find the container with id ef62dc6212256618ea4acf3fe644979b560a36ede888abfd9cd3bf529b552dce
Sep 11 18:35:52.061103 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.061046    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"]
Sep 11 18:35:52.072777 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:35:52.072754    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd790aa_d848_4a74_aded_92c7d67d122d.slice/crio-4d408870fea56c606cb6c61c8f94b4333459d53fe1b1315b52ce8b112f791c7d WatchSource:0}: Error finding container 4d408870fea56c606cb6c61c8f94b4333459d53fe1b1315b52ce8b112f791c7d: Status 404 returned error can't find the container with id 4d408870fea56c606cb6c61c8f94b4333459d53fe1b1315b52ce8b112f791c7d
Sep 11 18:35:52.272630 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.272588    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc" event={"ID":"5bd790aa-d848-4a74-aded-92c7d67d122d","Type":"ContainerStarted","Data":"34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6"}
Sep 11 18:35:52.272835 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.272782    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:35:52.272835 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.272811    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc" event={"ID":"5bd790aa-d848-4a74-aded-92c7d67d122d","Type":"ContainerStarted","Data":"4d408870fea56c606cb6c61c8f94b4333459d53fe1b1315b52ce8b112f791c7d"}
Sep 11 18:35:52.274214 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.274184    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj" event={"ID":"d93b6c95-1f08-438c-9979-9363303e8818","Type":"ContainerStarted","Data":"4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291"}
Sep 11 18:35:52.274323 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.274229    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj" event={"ID":"d93b6c95-1f08-438c-9979-9363303e8818","Type":"ContainerStarted","Data":"ef62dc6212256618ea4acf3fe644979b560a36ede888abfd9cd3bf529b552dce"}
Sep 11 18:35:52.274372 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.274309    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:35:52.299845 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.299744    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc" podStartSLOduration=1.299724737 podStartE2EDuration="1.299724737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:35:52.295389466 +0000 UTC m=+1215.330606136" watchObservedRunningTime="2026-09-11 18:35:52.299724737 +0000 UTC m=+1215.334941412"
Sep 11 18:35:52.320352 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:35:52.320305    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj" podStartSLOduration=1.320289071 podStartE2EDuration="1.320289071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:35:52.319568602 +0000 UTC m=+1215.354785273" watchObservedRunningTime="2026-09-11 18:35:52.320289071 +0000 UTC m=+1215.355505740"
Sep 11 18:36:03.282521 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:03.282488    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:36:03.282896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:03.282652    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:36:35.387128 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.387036    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"]
Sep 11 18:36:35.391009 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.390985    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:36:35.394739 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.394710    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-51815453\""
Sep 11 18:36:35.394848 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.394796    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-51815453-dockercfg-m8dkq\""
Sep 11 18:36:35.424325 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.424290    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"]
Sep 11 18:36:35.460858 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.460824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/195e1656-c516-40b2-a3ec-67fbec8328f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-51815453-59b646665d-66722\" (UID: \"195e1656-c516-40b2-a3ec-67fbec8328f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:36:35.461132 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.460991    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzprb\" (UniqueName: \"kubernetes.io/projected/195e1656-c516-40b2-a3ec-67fbec8328f8-kube-api-access-rzprb\") pod \"payload-pre-processing-e2e-mt-51815453-59b646665d-66722\" (UID: \"195e1656-c516-40b2-a3ec-67fbec8328f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:36:35.562233 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.562200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/195e1656-c516-40b2-a3ec-67fbec8328f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-51815453-59b646665d-66722\" (UID: \"195e1656-c516-40b2-a3ec-67fbec8328f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:36:35.562394 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.562350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzprb\" (UniqueName: \"kubernetes.io/projected/195e1656-c516-40b2-a3ec-67fbec8328f8-kube-api-access-rzprb\") pod \"payload-pre-processing-e2e-mt-51815453-59b646665d-66722\" (UID: \"195e1656-c516-40b2-a3ec-67fbec8328f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:36:35.562921 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.562902    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/195e1656-c516-40b2-a3ec-67fbec8328f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-51815453-59b646665d-66722\" (UID: \"195e1656-c516-40b2-a3ec-67fbec8328f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:36:35.578356 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.578327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzprb\" (UniqueName: \"kubernetes.io/projected/195e1656-c516-40b2-a3ec-67fbec8328f8-kube-api-access-rzprb\") pod \"payload-pre-processing-e2e-mt-51815453-59b646665d-66722\" (UID: \"195e1656-c516-40b2-a3ec-67fbec8328f8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:36:35.701734 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.701665    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:36:35.997396 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:35.997373    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"]
Sep 11 18:36:35.999970 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:36:35.999926    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195e1656_c516_40b2_a3ec_67fbec8328f8.slice/crio-cde41163d36c6a89798c1597d588ae48ce136848bbfb32ced87cd2396430913e WatchSource:0}: Error finding container cde41163d36c6a89798c1597d588ae48ce136848bbfb32ced87cd2396430913e: Status 404 returned error can't find the container with id cde41163d36c6a89798c1597d588ae48ce136848bbfb32ced87cd2396430913e
Sep 11 18:36:36.528358 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:36.528313    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722" event={"ID":"195e1656-c516-40b2-a3ec-67fbec8328f8","Type":"ContainerStarted","Data":"915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947"}
Sep 11 18:36:36.528879 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:36.528379    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722" event={"ID":"195e1656-c516-40b2-a3ec-67fbec8328f8","Type":"ContainerStarted","Data":"cde41163d36c6a89798c1597d588ae48ce136848bbfb32ced87cd2396430913e"}
Sep 11 18:36:36.528879 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:36.528501    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:36:36.563251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:36.563204    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722" podStartSLOduration=1.56318759 podStartE2EDuration="1.56318759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:36:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:36:36.562962129 +0000 UTC m=+1259.598178799" watchObservedRunningTime="2026-09-11 18:36:36.56318759 +0000 UTC m=+1259.598404288"
Sep 11 18:36:47.537031 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:36:47.536996    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:37:04.934337 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:04.934302    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"]
Sep 11 18:37:04.936928 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:04.936911    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"
Sep 11 18:37:04.939593 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:04.939541    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:37:04.939764 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:04.939744    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:37:04.953632 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:04.953605    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"]
Sep 11 18:37:04.954205 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:04.954187    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73356d2-53bf-4d49-9972-20dae72683bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-51815453-vgx8m\" (UID: \"b73356d2-53bf-4d49-9972-20dae72683bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"
Sep 11 18:37:05.055426 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:05.055388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73356d2-53bf-4d49-9972-20dae72683bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-51815453-vgx8m\" (UID: \"b73356d2-53bf-4d49-9972-20dae72683bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"
Sep 11 18:37:05.056152 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:05.056128    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73356d2-53bf-4d49-9972-20dae72683bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-51815453-vgx8m\" (UID: \"b73356d2-53bf-4d49-9972-20dae72683bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"
Sep 11 18:37:05.247516 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:05.247413    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"
Sep 11 18:37:05.526848 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:05.526818    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"]
Sep 11 18:37:05.528618 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:37:05.528589    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73356d2_53bf_4d49_9972_20dae72683bc.slice/crio-4e74674571af981d0f6e6b18fdb5e1d434a0fb9587305e10f5d2cab2d5bbe583 WatchSource:0}: Error finding container 4e74674571af981d0f6e6b18fdb5e1d434a0fb9587305e10f5d2cab2d5bbe583: Status 404 returned error can't find the container with id 4e74674571af981d0f6e6b18fdb5e1d434a0fb9587305e10f5d2cab2d5bbe583
Sep 11 18:37:05.696061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:05.696027    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m" event={"ID":"b73356d2-53bf-4d49-9972-20dae72683bc","Type":"ContainerStarted","Data":"4e74674571af981d0f6e6b18fdb5e1d434a0fb9587305e10f5d2cab2d5bbe583"}
Sep 11 18:37:06.703032 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:06.702990    2578 generic.go:356] "Generic (PLEG): container finished" podID="b73356d2-53bf-4d49-9972-20dae72683bc" containerID="76dfa66ee348fb0fa1e463e138804cd7752345ce0935f92591c69357d654d358" exitCode=0
Sep 11 18:37:06.703476 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:06.703033    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m" event={"ID":"b73356d2-53bf-4d49-9972-20dae72683bc","Type":"ContainerDied","Data":"76dfa66ee348fb0fa1e463e138804cd7752345ce0935f92591c69357d654d358"}
Sep 11 18:37:07.853261 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:07.853236    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"
Sep 11 18:37:07.881489 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:07.881457    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73356d2-53bf-4d49-9972-20dae72683bc-maas-api-service-ca\") pod \"b73356d2-53bf-4d49-9972-20dae72683bc\" (UID: \"b73356d2-53bf-4d49-9972-20dae72683bc\") "
Sep 11 18:37:07.881917 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:07.881887    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b73356d2-53bf-4d49-9972-20dae72683bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b73356d2-53bf-4d49-9972-20dae72683bc" (UID: "b73356d2-53bf-4d49-9972-20dae72683bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:37:07.983112 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:07.982999    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73356d2-53bf-4d49-9972-20dae72683bc-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:08.714977 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:08.714944    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"
Sep 11 18:37:08.715170 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:08.714996    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m" event={"ID":"b73356d2-53bf-4d49-9972-20dae72683bc","Type":"ContainerDied","Data":"4e74674571af981d0f6e6b18fdb5e1d434a0fb9587305e10f5d2cab2d5bbe583"}
Sep 11 18:37:08.715170 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:08.715047    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e74674571af981d0f6e6b18fdb5e1d434a0fb9587305e10f5d2cab2d5bbe583"
Sep 11 18:37:15.251546 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.251506    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"]
Sep 11 18:37:15.251934 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.251918    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b73356d2-53bf-4d49-9972-20dae72683bc" containerName="revoke-keys"
Sep 11 18:37:15.251934 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.251930    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b73356d2-53bf-4d49-9972-20dae72683bc" containerName="revoke-keys"
Sep 11 18:37:15.252065 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.252028    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b73356d2-53bf-4d49-9972-20dae72683bc" containerName="revoke-keys"
Sep 11 18:37:15.254377 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.254359    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.257268 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.257243    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-a228c4\"/\"kube-root-ca.crt\""
Sep 11 18:37:15.258092 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.258075    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-a228c4\"/\"openshift-service-ca.crt\""
Sep 11 18:37:15.258210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.258130    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-a228c4\"/\"default-dockercfg-x54vv\""
Sep 11 18:37:15.258210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.258171    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-a228c4\"/\"explicit-model-a228c4-kserve-self-signed-certs\""
Sep 11 18:37:15.271567 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.271542    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"]
Sep 11 18:37:15.357097 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.357061    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-866hg\" (UniqueName: \"kubernetes.io/projected/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-kube-api-access-866hg\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.357281 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.357112    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tls-certs\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.357281 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.357252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-dshm\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.357393 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.357296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-home\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.357393 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.357328    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-model-cache\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.357393 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.357345    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tmp-dir\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.458120 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.458083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-866hg\" (UniqueName: \"kubernetes.io/projected/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-kube-api-access-866hg\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.458325 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.458161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tls-certs\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.458404 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.458333    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-dshm\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.458404 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.458388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-home\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.458519 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.458453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-model-cache\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.458519 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.458480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tmp-dir\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.458812 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.458786    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-home\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.458940 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.458875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-model-cache\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.459024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.459008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tmp-dir\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.460709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.460665    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-dshm\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.460811 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.460794    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tls-certs\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.467330 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.467306    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-866hg\" (UniqueName: \"kubernetes.io/projected/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-kube-api-access-866hg\") pod \"explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") " pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.563556 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.563468    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:15.660733 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.659867    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"]
Sep 11 18:37:15.661074 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.660806    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722" podUID="195e1656-c516-40b2-a3ec-67fbec8328f8" containerName="payload-pre-processing" containerID="cri-o://915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947" gracePeriod=30
Sep 11 18:37:15.858566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.858496    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"]
Sep 11 18:37:15.860356 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:37:15.860325    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07ed5d86_bcf9_4931_a7f9_cd6d693519f7.slice/crio-5a50edb901083cc280fe609284ec614bf108a42a3bc2be79f9227d9a4d099c10 WatchSource:0}: Error finding container 5a50edb901083cc280fe609284ec614bf108a42a3bc2be79f9227d9a4d099c10: Status 404 returned error can't find the container with id 5a50edb901083cc280fe609284ec614bf108a42a3bc2be79f9227d9a4d099c10
Sep 11 18:37:15.871511 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.871171    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"]
Sep 11 18:37:15.871511 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.871223    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51815453-vgx8m"]
Sep 11 18:37:15.917155 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.917119    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:37:15.963473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.963437    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/195e1656-c516-40b2-a3ec-67fbec8328f8-plugins-config-volume\") pod \"195e1656-c516-40b2-a3ec-67fbec8328f8\" (UID: \"195e1656-c516-40b2-a3ec-67fbec8328f8\") "
Sep 11 18:37:15.963632 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.963512    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzprb\" (UniqueName: \"kubernetes.io/projected/195e1656-c516-40b2-a3ec-67fbec8328f8-kube-api-access-rzprb\") pod \"195e1656-c516-40b2-a3ec-67fbec8328f8\" (UID: \"195e1656-c516-40b2-a3ec-67fbec8328f8\") "
Sep 11 18:37:15.963913 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.963883    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/195e1656-c516-40b2-a3ec-67fbec8328f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "195e1656-c516-40b2-a3ec-67fbec8328f8" (UID: "195e1656-c516-40b2-a3ec-67fbec8328f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:37:15.964014 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.963977    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/195e1656-c516-40b2-a3ec-67fbec8328f8-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:15.966193 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:15.966161    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/195e1656-c516-40b2-a3ec-67fbec8328f8-kube-api-access-rzprb" (OuterVolumeSpecName: "kube-api-access-rzprb") pod "195e1656-c516-40b2-a3ec-67fbec8328f8" (UID: "195e1656-c516-40b2-a3ec-67fbec8328f8"). InnerVolumeSpecName "kube-api-access-rzprb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:37:16.064982 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.064944    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzprb\" (UniqueName: \"kubernetes.io/projected/195e1656-c516-40b2-a3ec-67fbec8328f8-kube-api-access-rzprb\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:16.764010 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.763917    2578 generic.go:356] "Generic (PLEG): container finished" podID="195e1656-c516-40b2-a3ec-67fbec8328f8" containerID="915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947" exitCode=0
Sep 11 18:37:16.764010 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.763976    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"
Sep 11 18:37:16.764521 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.764008    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722" event={"ID":"195e1656-c516-40b2-a3ec-67fbec8328f8","Type":"ContainerDied","Data":"915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947"}
Sep 11 18:37:16.764521 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.764065    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722" event={"ID":"195e1656-c516-40b2-a3ec-67fbec8328f8","Type":"ContainerDied","Data":"cde41163d36c6a89798c1597d588ae48ce136848bbfb32ced87cd2396430913e"}
Sep 11 18:37:16.764521 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.764093    2578 scope.go:117] "RemoveContainer" containerID="915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947"
Sep 11 18:37:16.765956 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.765926    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk" event={"ID":"07ed5d86-bcf9-4931-a7f9-cd6d693519f7","Type":"ContainerStarted","Data":"f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58"}
Sep 11 18:37:16.766044 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.765974    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk" event={"ID":"07ed5d86-bcf9-4931-a7f9-cd6d693519f7","Type":"ContainerStarted","Data":"5a50edb901083cc280fe609284ec614bf108a42a3bc2be79f9227d9a4d099c10"}
Sep 11 18:37:16.773316 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.773291    2578 scope.go:117] "RemoveContainer" containerID="915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947"
Sep 11 18:37:16.773575 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:37:16.773555    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947\": container with ID starting with 915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947 not found: ID does not exist" containerID="915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947"
Sep 11 18:37:16.773634 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.773586    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947"} err="failed to get container status \"915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947\": rpc error: code = NotFound desc = could not find container \"915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947\": container with ID starting with 915da3ce3b6b93d6086b9fcfcd0f0a29ddfcab4010d49d02085a599366e40947 not found: ID does not exist"
Sep 11 18:37:16.871873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.871815    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk" podStartSLOduration=1.8717982499999999 podStartE2EDuration="1.87179825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:37:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:37:16.84780603 +0000 UTC m=+1299.883022701" watchObservedRunningTime="2026-09-11 18:37:16.87179825 +0000 UTC m=+1299.907014919"
Sep 11 18:37:16.903611 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.903579    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"]
Sep 11 18:37:16.910471 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:16.910446    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51815453-59b646665d-66722"]
Sep 11 18:37:17.415569 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:17.415535    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="195e1656-c516-40b2-a3ec-67fbec8328f8" path="/var/lib/kubelet/pods/195e1656-c516-40b2-a3ec-67fbec8328f8/volumes"
Sep 11 18:37:17.416388 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:17.416350    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b73356d2-53bf-4d49-9972-20dae72683bc" path="/var/lib/kubelet/pods/b73356d2-53bf-4d49-9972-20dae72683bc/volumes"
Sep 11 18:37:25.564167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:25.564131    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:25.564670 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:25.564186    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:25.578102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:25.578074    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:25.828957 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:25.828869    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:36.444264 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.444229    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"]
Sep 11 18:37:36.576271 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.576225    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"]
Sep 11 18:37:36.576788 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.576767    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="195e1656-c516-40b2-a3ec-67fbec8328f8" containerName="payload-pre-processing"
Sep 11 18:37:36.576788 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.576783    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="195e1656-c516-40b2-a3ec-67fbec8328f8" containerName="payload-pre-processing"
Sep 11 18:37:36.577014 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.576880    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="195e1656-c516-40b2-a3ec-67fbec8328f8" containerName="payload-pre-processing"
Sep 11 18:37:36.581670 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.581653    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"
Sep 11 18:37:36.584024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.583988    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:37:36.584216 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.584191    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:37:36.587927 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.587901    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"]
Sep 11 18:37:36.671327 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.671281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9bee43-57c6-41c5-811b-963804617889-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k\" (UID: \"4b9bee43-57c6-41c5-811b-963804617889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"
Sep 11 18:37:36.772376 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.772277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9bee43-57c6-41c5-811b-963804617889-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k\" (UID: \"4b9bee43-57c6-41c5-811b-963804617889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"
Sep 11 18:37:36.772981 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.772960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9bee43-57c6-41c5-811b-963804617889-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k\" (UID: \"4b9bee43-57c6-41c5-811b-963804617889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"
Sep 11 18:37:36.895433 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:36.895400    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"
Sep 11 18:37:37.172875 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:37.172848    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"]
Sep 11 18:37:37.174937 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:37:37.174907    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b9bee43_57c6_41c5_811b_963804617889.slice/crio-e88ef342bb8cd7f055459abe82f9b91d74dbb23fdee2b36332ad00fe50e019c3 WatchSource:0}: Error finding container e88ef342bb8cd7f055459abe82f9b91d74dbb23fdee2b36332ad00fe50e019c3: Status 404 returned error can't find the container with id e88ef342bb8cd7f055459abe82f9b91d74dbb23fdee2b36332ad00fe50e019c3
Sep 11 18:37:37.889411 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:37.889310    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k" event={"ID":"4b9bee43-57c6-41c5-811b-963804617889","Type":"ContainerStarted","Data":"d31cffa7e44e9e78b0e978e57f02cf1dfe3a9818acb5f610e4cc3012bf2103c7"}
Sep 11 18:37:37.889411 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:37.889363    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k" event={"ID":"4b9bee43-57c6-41c5-811b-963804617889","Type":"ContainerStarted","Data":"e88ef342bb8cd7f055459abe82f9b91d74dbb23fdee2b36332ad00fe50e019c3"}
Sep 11 18:37:37.911178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:37.911114    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k" podStartSLOduration=1.911092416 podStartE2EDuration="1.911092416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:37:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:37:37.910518614 +0000 UTC m=+1320.945735285" watchObservedRunningTime="2026-09-11 18:37:37.911092416 +0000 UTC m=+1320.946309087"
Sep 11 18:37:38.896460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:38.896423    2578 generic.go:356] "Generic (PLEG): container finished" podID="4b9bee43-57c6-41c5-811b-963804617889" containerID="d31cffa7e44e9e78b0e978e57f02cf1dfe3a9818acb5f610e4cc3012bf2103c7" exitCode=0
Sep 11 18:37:38.896902 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:38.896489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k" event={"ID":"4b9bee43-57c6-41c5-811b-963804617889","Type":"ContainerDied","Data":"d31cffa7e44e9e78b0e978e57f02cf1dfe3a9818acb5f610e4cc3012bf2103c7"}
Sep 11 18:37:40.046440 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:40.046415    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"
Sep 11 18:37:40.209928 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:40.209892    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9bee43-57c6-41c5-811b-963804617889-maas-api-service-ca\") pod \"4b9bee43-57c6-41c5-811b-963804617889\" (UID: \"4b9bee43-57c6-41c5-811b-963804617889\") "
Sep 11 18:37:40.210261 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:40.210235    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b9bee43-57c6-41c5-811b-963804617889-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b9bee43-57c6-41c5-811b-963804617889" (UID: "4b9bee43-57c6-41c5-811b-963804617889"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:37:40.311514 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:40.311475    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b9bee43-57c6-41c5-811b-963804617889-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:40.910530 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:40.910499    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"
Sep 11 18:37:40.910530 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:40.910513    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k" event={"ID":"4b9bee43-57c6-41c5-811b-963804617889","Type":"ContainerDied","Data":"e88ef342bb8cd7f055459abe82f9b91d74dbb23fdee2b36332ad00fe50e019c3"}
Sep 11 18:37:40.910792 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:40.910556    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e88ef342bb8cd7f055459abe82f9b91d74dbb23fdee2b36332ad00fe50e019c3"
Sep 11 18:37:44.377074 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.377026    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"]
Sep 11 18:37:44.377481 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.377456    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b9bee43-57c6-41c5-811b-963804617889" containerName="revoke-keys"
Sep 11 18:37:44.377481 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.377468    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b9bee43-57c6-41c5-811b-963804617889" containerName="revoke-keys"
Sep 11 18:37:44.377558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.377529    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b9bee43-57c6-41c5-811b-963804617889" containerName="revoke-keys"
Sep 11 18:37:44.380862 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.380841    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:44.383542 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.383519    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-72c98d\""
Sep 11 18:37:44.395102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.395076    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"]
Sep 11 18:37:44.397918 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.397894    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"]
Sep 11 18:37:44.408201 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.408176    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:44.414211 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.414188    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-72c98d-dockercfg-sfdg8\""
Sep 11 18:37:44.417991 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.417964    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"]
Sep 11 18:37:44.445748 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.445720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22d6s\" (UniqueName: \"kubernetes.io/projected/640bcbc4-85c5-4895-acbe-f2d5737193d4-kube-api-access-22d6s\") pod \"payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb\" (UID: \"640bcbc4-85c5-4895-acbe-f2d5737193d4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:44.445893 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.445814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/640bcbc4-85c5-4895-acbe-f2d5737193d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb\" (UID: \"640bcbc4-85c5-4895-acbe-f2d5737193d4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:44.445893 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.445834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c194fbca-c1ac-4441-9ab4-1f902110b978-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf\" (UID: \"c194fbca-c1ac-4441-9ab4-1f902110b978\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:44.445971 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.445910    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhcvq\" (UniqueName: \"kubernetes.io/projected/c194fbca-c1ac-4441-9ab4-1f902110b978-kube-api-access-hhcvq\") pod \"payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf\" (UID: \"c194fbca-c1ac-4441-9ab4-1f902110b978\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:44.546364 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.546323    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/640bcbc4-85c5-4895-acbe-f2d5737193d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb\" (UID: \"640bcbc4-85c5-4895-acbe-f2d5737193d4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:44.546364 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.546364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c194fbca-c1ac-4441-9ab4-1f902110b978-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf\" (UID: \"c194fbca-c1ac-4441-9ab4-1f902110b978\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:44.546576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.546398    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhcvq\" (UniqueName: \"kubernetes.io/projected/c194fbca-c1ac-4441-9ab4-1f902110b978-kube-api-access-hhcvq\") pod \"payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf\" (UID: \"c194fbca-c1ac-4441-9ab4-1f902110b978\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:44.546576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.546430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22d6s\" (UniqueName: \"kubernetes.io/projected/640bcbc4-85c5-4895-acbe-f2d5737193d4-kube-api-access-22d6s\") pod \"payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb\" (UID: \"640bcbc4-85c5-4895-acbe-f2d5737193d4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:44.547319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.547297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c194fbca-c1ac-4441-9ab4-1f902110b978-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf\" (UID: \"c194fbca-c1ac-4441-9ab4-1f902110b978\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:44.547372 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.547297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/640bcbc4-85c5-4895-acbe-f2d5737193d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb\" (UID: \"640bcbc4-85c5-4895-acbe-f2d5737193d4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:44.556118 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.556093    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhcvq\" (UniqueName: \"kubernetes.io/projected/c194fbca-c1ac-4441-9ab4-1f902110b978-kube-api-access-hhcvq\") pod \"payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf\" (UID: \"c194fbca-c1ac-4441-9ab4-1f902110b978\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:44.557096 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.557075    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22d6s\" (UniqueName: \"kubernetes.io/projected/640bcbc4-85c5-4895-acbe-f2d5737193d4-kube-api-access-22d6s\") pod \"payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb\" (UID: \"640bcbc4-85c5-4895-acbe-f2d5737193d4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:44.691015 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.690978    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:44.722933 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.722900    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:44.978064 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:44.978023    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"]
Sep 11 18:37:44.979118 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:37:44.979092    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod640bcbc4_85c5_4895_acbe_f2d5737193d4.slice/crio-1fcfd6b75a2d272e5f2a5478c60a65d9d44e66e0991e9194ccbe1f2275387aaf WatchSource:0}: Error finding container 1fcfd6b75a2d272e5f2a5478c60a65d9d44e66e0991e9194ccbe1f2275387aaf: Status 404 returned error can't find the container with id 1fcfd6b75a2d272e5f2a5478c60a65d9d44e66e0991e9194ccbe1f2275387aaf
Sep 11 18:37:45.033997 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:45.033971    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"]
Sep 11 18:37:45.034385 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:37:45.034358    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc194fbca_c1ac_4441_9ab4_1f902110b978.slice/crio-ccea27929357fbb4153a89562bd5a477d525f97b85a2f6b325a10e4fab1041d4 WatchSource:0}: Error finding container ccea27929357fbb4153a89562bd5a477d525f97b85a2f6b325a10e4fab1041d4: Status 404 returned error can't find the container with id ccea27929357fbb4153a89562bd5a477d525f97b85a2f6b325a10e4fab1041d4
Sep 11 18:37:45.939505 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:45.939469    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf" event={"ID":"c194fbca-c1ac-4441-9ab4-1f902110b978","Type":"ContainerStarted","Data":"35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b"}
Sep 11 18:37:45.939965 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:45.939528    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf" event={"ID":"c194fbca-c1ac-4441-9ab4-1f902110b978","Type":"ContainerStarted","Data":"ccea27929357fbb4153a89562bd5a477d525f97b85a2f6b325a10e4fab1041d4"}
Sep 11 18:37:45.939965 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:45.939558    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:45.941188 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:45.941160    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb" event={"ID":"640bcbc4-85c5-4895-acbe-f2d5737193d4","Type":"ContainerStarted","Data":"759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b"}
Sep 11 18:37:45.941315 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:45.941210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb" event={"ID":"640bcbc4-85c5-4895-acbe-f2d5737193d4","Type":"ContainerStarted","Data":"1fcfd6b75a2d272e5f2a5478c60a65d9d44e66e0991e9194ccbe1f2275387aaf"}
Sep 11 18:37:45.941315 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:45.941249    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:45.960260 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:45.960215    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf" podStartSLOduration=1.960199668 podStartE2EDuration="1.960199668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:37:45.958251698 +0000 UTC m=+1328.993468367" watchObservedRunningTime="2026-09-11 18:37:45.960199668 +0000 UTC m=+1328.995416338"
Sep 11 18:37:45.978968 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:45.978895    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb" podStartSLOduration=1.978877952 podStartE2EDuration="1.978877952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:37:45.97538882 +0000 UTC m=+1329.010605488" watchObservedRunningTime="2026-09-11 18:37:45.978877952 +0000 UTC m=+1329.014094622"
Sep 11 18:37:48.572497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.572454    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"]
Sep 11 18:37:48.573046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.572722    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc" podUID="5bd790aa-d848-4a74-aded-92c7d67d122d" containerName="payload-processing" containerID="cri-o://34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6" gracePeriod=30
Sep 11 18:37:48.583483 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.583450    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"]
Sep 11 18:37:48.583805 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.583773    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj" podUID="d93b6c95-1f08-438c-9979-9363303e8818" containerName="payload-pre-processing" containerID="cri-o://4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291" gracePeriod=30
Sep 11 18:37:48.783937 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.783900    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"]
Sep 11 18:37:48.788601 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.788547    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a228c4-4nb2k"]
Sep 11 18:37:48.878190 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.878162    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:37:48.881030 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.881009    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:37:48.960953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.960920    2578 generic.go:356] "Generic (PLEG): container finished" podID="d93b6c95-1f08-438c-9979-9363303e8818" containerID="4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291" exitCode=0
Sep 11 18:37:48.961126 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.960970    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"
Sep 11 18:37:48.961126 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.961002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj" event={"ID":"d93b6c95-1f08-438c-9979-9363303e8818","Type":"ContainerDied","Data":"4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291"}
Sep 11 18:37:48.961126 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.961053    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj" event={"ID":"d93b6c95-1f08-438c-9979-9363303e8818","Type":"ContainerDied","Data":"ef62dc6212256618ea4acf3fe644979b560a36ede888abfd9cd3bf529b552dce"}
Sep 11 18:37:48.961126 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.961076    2578 scope.go:117] "RemoveContainer" containerID="4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291"
Sep 11 18:37:48.962423 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.962361    2578 generic.go:356] "Generic (PLEG): container finished" podID="5bd790aa-d848-4a74-aded-92c7d67d122d" containerID="34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6" exitCode=0
Sep 11 18:37:48.962423 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.962400    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"
Sep 11 18:37:48.962597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.962402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc" event={"ID":"5bd790aa-d848-4a74-aded-92c7d67d122d","Type":"ContainerDied","Data":"34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6"}
Sep 11 18:37:48.962597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.962518    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc" event={"ID":"5bd790aa-d848-4a74-aded-92c7d67d122d","Type":"ContainerDied","Data":"4d408870fea56c606cb6c61c8f94b4333459d53fe1b1315b52ce8b112f791c7d"}
Sep 11 18:37:48.973068 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.973042    2578 scope.go:117] "RemoveContainer" containerID="4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291"
Sep 11 18:37:48.973390 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:37:48.973362    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291\": container with ID starting with 4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291 not found: ID does not exist" containerID="4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291"
Sep 11 18:37:48.973449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.973392    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291"} err="failed to get container status \"4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291\": rpc error: code = NotFound desc = could not find container \"4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291\": container with ID starting with 4b9c8db786cf1d120a062bc73aa24e9f38447f04695fd49ac3a000de912fd291 not found: ID does not exist"
Sep 11 18:37:48.973449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.973408    2578 scope.go:117] "RemoveContainer" containerID="34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6"
Sep 11 18:37:48.981941 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.981925    2578 scope.go:117] "RemoveContainer" containerID="34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6"
Sep 11 18:37:48.982203 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:37:48.982184    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6\": container with ID starting with 34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6 not found: ID does not exist" containerID="34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6"
Sep 11 18:37:48.982244 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.982217    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6"} err="failed to get container status \"34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6\": rpc error: code = NotFound desc = could not find container \"34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6\": container with ID starting with 34cf1c751fad71588e6791a3e90748022c6e98d19f5ac7f0ac2bcf058f620ca6 not found: ID does not exist"
Sep 11 18:37:48.994576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.994554    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29rwh\" (UniqueName: \"kubernetes.io/projected/d93b6c95-1f08-438c-9979-9363303e8818-kube-api-access-29rwh\") pod \"d93b6c95-1f08-438c-9979-9363303e8818\" (UID: \"d93b6c95-1f08-438c-9979-9363303e8818\") "
Sep 11 18:37:48.994669 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.994638    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x24gh\" (UniqueName: \"kubernetes.io/projected/5bd790aa-d848-4a74-aded-92c7d67d122d-kube-api-access-x24gh\") pod \"5bd790aa-d848-4a74-aded-92c7d67d122d\" (UID: \"5bd790aa-d848-4a74-aded-92c7d67d122d\") "
Sep 11 18:37:48.994771 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.994726    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d93b6c95-1f08-438c-9979-9363303e8818-plugins-config-volume\") pod \"d93b6c95-1f08-438c-9979-9363303e8818\" (UID: \"d93b6c95-1f08-438c-9979-9363303e8818\") "
Sep 11 18:37:48.994828 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.994797    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bd790aa-d848-4a74-aded-92c7d67d122d-plugins-config-volume\") pod \"5bd790aa-d848-4a74-aded-92c7d67d122d\" (UID: \"5bd790aa-d848-4a74-aded-92c7d67d122d\") "
Sep 11 18:37:48.995109 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.995085    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d93b6c95-1f08-438c-9979-9363303e8818-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d93b6c95-1f08-438c-9979-9363303e8818" (UID: "d93b6c95-1f08-438c-9979-9363303e8818"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:37:48.995211 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.995136    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bd790aa-d848-4a74-aded-92c7d67d122d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5bd790aa-d848-4a74-aded-92c7d67d122d" (UID: "5bd790aa-d848-4a74-aded-92c7d67d122d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:37:48.997167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.997147    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bd790aa-d848-4a74-aded-92c7d67d122d-kube-api-access-x24gh" (OuterVolumeSpecName: "kube-api-access-x24gh") pod "5bd790aa-d848-4a74-aded-92c7d67d122d" (UID: "5bd790aa-d848-4a74-aded-92c7d67d122d"). InnerVolumeSpecName "kube-api-access-x24gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:37:48.997256 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:48.997161    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d93b6c95-1f08-438c-9979-9363303e8818-kube-api-access-29rwh" (OuterVolumeSpecName: "kube-api-access-29rwh") pod "d93b6c95-1f08-438c-9979-9363303e8818" (UID: "d93b6c95-1f08-438c-9979-9363303e8818"). InnerVolumeSpecName "kube-api-access-29rwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:37:49.096566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.096535    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bd790aa-d848-4a74-aded-92c7d67d122d-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:49.096566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.096561    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29rwh\" (UniqueName: \"kubernetes.io/projected/d93b6c95-1f08-438c-9979-9363303e8818-kube-api-access-29rwh\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:49.096566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.096570    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x24gh\" (UniqueName: \"kubernetes.io/projected/5bd790aa-d848-4a74-aded-92c7d67d122d-kube-api-access-x24gh\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:49.096808 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.096578    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d93b6c95-1f08-438c-9979-9363303e8818-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:49.303954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.303921    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"]
Sep 11 18:37:49.305984 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.305956    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a228c4-6fb6d4bf94-snpdj"]
Sep 11 18:37:49.317117 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.317087    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"]
Sep 11 18:37:49.321265 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.321238    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a228c4-7686c9dbd9-h8dfc"]
Sep 11 18:37:49.410532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.410505    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b9bee43-57c6-41c5-811b-963804617889" path="/var/lib/kubelet/pods/4b9bee43-57c6-41c5-811b-963804617889/volumes"
Sep 11 18:37:49.410922 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.410908    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bd790aa-d848-4a74-aded-92c7d67d122d" path="/var/lib/kubelet/pods/5bd790aa-d848-4a74-aded-92c7d67d122d/volumes"
Sep 11 18:37:49.411236 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:49.411225    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d93b6c95-1f08-438c-9979-9363303e8818" path="/var/lib/kubelet/pods/d93b6c95-1f08-438c-9979-9363303e8818/volumes"
Sep 11 18:37:51.463617 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.463582    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk" podUID="07ed5d86-bcf9-4931-a7f9-cd6d693519f7" containerName="main" containerID="cri-o://f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58" gracePeriod=45
Sep 11 18:37:51.713578 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.713550    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:51.825437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.825338    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tmp-dir\") pod \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") "
Sep 11 18:37:51.825437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.825379    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-model-cache\") pod \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") "
Sep 11 18:37:51.825437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.825436    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-dshm\") pod \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") "
Sep 11 18:37:51.825773 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.825455    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tls-certs\") pod \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") "
Sep 11 18:37:51.825773 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.825494    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-866hg\" (UniqueName: \"kubernetes.io/projected/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-kube-api-access-866hg\") pod \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") "
Sep 11 18:37:51.825773 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.825523    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-home\") pod \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\" (UID: \"07ed5d86-bcf9-4931-a7f9-cd6d693519f7\") "
Sep 11 18:37:51.825773 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.825758    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07ed5d86-bcf9-4931-a7f9-cd6d693519f7" (UID: "07ed5d86-bcf9-4931-a7f9-cd6d693519f7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:37:51.825976 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.825777    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-model-cache" (OuterVolumeSpecName: "model-cache") pod "07ed5d86-bcf9-4931-a7f9-cd6d693519f7" (UID: "07ed5d86-bcf9-4931-a7f9-cd6d693519f7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:37:51.825976 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.825869    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-home" (OuterVolumeSpecName: "home") pod "07ed5d86-bcf9-4931-a7f9-cd6d693519f7" (UID: "07ed5d86-bcf9-4931-a7f9-cd6d693519f7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:37:51.826081 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.826034    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tmp-dir\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:51.826081 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.826054    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-model-cache\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:51.826081 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.826069    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-home\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:51.827909 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.827881    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07ed5d86-bcf9-4931-a7f9-cd6d693519f7" (UID: "07ed5d86-bcf9-4931-a7f9-cd6d693519f7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:37:51.827909 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.827894    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-dshm" (OuterVolumeSpecName: "dshm") pod "07ed5d86-bcf9-4931-a7f9-cd6d693519f7" (UID: "07ed5d86-bcf9-4931-a7f9-cd6d693519f7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:37:51.827909 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.827903    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-kube-api-access-866hg" (OuterVolumeSpecName: "kube-api-access-866hg") pod "07ed5d86-bcf9-4931-a7f9-cd6d693519f7" (UID: "07ed5d86-bcf9-4931-a7f9-cd6d693519f7"). InnerVolumeSpecName "kube-api-access-866hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:37:51.927597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.927562    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-dshm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:51.927597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.927592    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-tls-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:51.927597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.927604    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-866hg\" (UniqueName: \"kubernetes.io/projected/07ed5d86-bcf9-4931-a7f9-cd6d693519f7-kube-api-access-866hg\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:37:51.982138 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.982103    2578 generic.go:356] "Generic (PLEG): container finished" podID="07ed5d86-bcf9-4931-a7f9-cd6d693519f7" containerID="f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58" exitCode=0
Sep 11 18:37:51.982310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.982147    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"
Sep 11 18:37:51.982310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.982193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk" event={"ID":"07ed5d86-bcf9-4931-a7f9-cd6d693519f7","Type":"ContainerDied","Data":"f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58"}
Sep 11 18:37:51.982310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.982238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk" event={"ID":"07ed5d86-bcf9-4931-a7f9-cd6d693519f7","Type":"ContainerDied","Data":"5a50edb901083cc280fe609284ec614bf108a42a3bc2be79f9227d9a4d099c10"}
Sep 11 18:37:51.982310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.982256    2578 scope.go:117] "RemoveContainer" containerID="f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58"
Sep 11 18:37:51.991805 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.991790    2578 scope.go:117] "RemoveContainer" containerID="f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58"
Sep 11 18:37:51.992089 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:37:51.992068    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58\": container with ID starting with f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58 not found: ID does not exist" containerID="f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58"
Sep 11 18:37:51.992168 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:51.992095    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58"} err="failed to get container status \"f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58\": rpc error: code = NotFound desc = could not find container \"f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58\": container with ID starting with f05f36ad98ded21c11274ff81ebb44dd480938f546bfb2c1f99a89d6ed8c4f58 not found: ID does not exist"
Sep 11 18:37:52.017001 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:52.016968    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"]
Sep 11 18:37:52.021256 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:52.021229    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-a228c4/explicit-model-a228c4-kserve-5f5dc8b59c-k7wvk"]
Sep 11 18:37:53.411038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:53.411003    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07ed5d86-bcf9-4931-a7f9-cd6d693519f7" path="/var/lib/kubelet/pods/07ed5d86-bcf9-4931-a7f9-cd6d693519f7/volumes"
Sep 11 18:37:56.503848 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.503810    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"]
Sep 11 18:37:56.504305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.504211    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ed5d86-bcf9-4931-a7f9-cd6d693519f7" containerName="main"
Sep 11 18:37:56.504305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.504223    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ed5d86-bcf9-4931-a7f9-cd6d693519f7" containerName="main"
Sep 11 18:37:56.504305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.504236    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd790aa-d848-4a74-aded-92c7d67d122d" containerName="payload-processing"
Sep 11 18:37:56.504305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.504242    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd790aa-d848-4a74-aded-92c7d67d122d" containerName="payload-processing"
Sep 11 18:37:56.504305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.504252    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d93b6c95-1f08-438c-9979-9363303e8818" containerName="payload-pre-processing"
Sep 11 18:37:56.504305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.504257    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d93b6c95-1f08-438c-9979-9363303e8818" containerName="payload-pre-processing"
Sep 11 18:37:56.504534 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.504327    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d93b6c95-1f08-438c-9979-9363303e8818" containerName="payload-pre-processing"
Sep 11 18:37:56.504534 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.504336    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ed5d86-bcf9-4931-a7f9-cd6d693519f7" containerName="main"
Sep 11 18:37:56.504534 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.504344    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bd790aa-d848-4a74-aded-92c7d67d122d" containerName="payload-processing"
Sep 11 18:37:56.508785 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.508760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.511315 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.511293    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-3b7488-openshift-default-dockercfg-58lvb\""
Sep 11 18:37:56.523389 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.523360    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"]
Sep 11 18:37:56.575825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.575789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-envoy\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.576015 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.575832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-credential-socket\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.576015 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.575869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-istio-token\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.576015 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.575903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpqvw\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-kube-api-access-gpqvw\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.576015 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.575931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-data\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.576167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.576014    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-socket\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.576167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.576041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/757ddd53-e60d-4a21-9676-702abca84956-istiod-ca-cert\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.576167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.576085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/757ddd53-e60d-4a21-9676-702abca84956-istio-podinfo\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.576167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.576113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-certs\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677208 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-data\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677208 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-socket\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677464 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/757ddd53-e60d-4a21-9676-702abca84956-istiod-ca-cert\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677464 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/757ddd53-e60d-4a21-9676-702abca84956-istio-podinfo\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677464 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-certs\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677464 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-envoy\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677464 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-credential-socket\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677464 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677418    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-istio-token\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677464 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677458    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpqvw\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-kube-api-access-gpqvw\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677857 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-socket\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.677857 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.677716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-data\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.678023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.678007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-certs\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.678156 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.678130    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-credential-socket\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.678593 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.678575    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/757ddd53-e60d-4a21-9676-702abca84956-istiod-ca-cert\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.680315 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.680287    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-envoy\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.680442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.680372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/757ddd53-e60d-4a21-9676-702abca84956-istio-podinfo\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.692309 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.692279    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-istio-token\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.694041 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.694012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpqvw\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-kube-api-access-gpqvw\") pod \"e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") " pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.819393 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.819067    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:56.950268 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.949925    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:37:56.950848 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:56.950815    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:37:57.112281 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:57.112251    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"]
Sep 11 18:37:57.114352 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:37:57.114317    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod757ddd53_e60d_4a21_9676_702abca84956.slice/crio-90961f7796466d0791389b417dcd94f59f6f906deffbe39ec2a342b41c982e7b WatchSource:0}: Error finding container 90961f7796466d0791389b417dcd94f59f6f906deffbe39ec2a342b41c982e7b: Status 404 returned error can't find the container with id 90961f7796466d0791389b417dcd94f59f6f906deffbe39ec2a342b41c982e7b
Sep 11 18:37:57.116767 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:57.116735    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:37:57.116872 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:57.116802    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:37:57.116872 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:57.116831    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:37:58.020144 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:58.020105    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7" event={"ID":"757ddd53-e60d-4a21-9676-702abca84956","Type":"ContainerStarted","Data":"aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a"}
Sep 11 18:37:58.020632 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:58.020190    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7" event={"ID":"757ddd53-e60d-4a21-9676-702abca84956","Type":"ContainerStarted","Data":"90961f7796466d0791389b417dcd94f59f6f906deffbe39ec2a342b41c982e7b"}
Sep 11 18:37:58.819802 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:58.819761    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:58.825110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:58.825079    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:58.856991 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:58.856936    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7" podStartSLOduration=2.856918212 podStartE2EDuration="2.856918212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:37:58.049986131 +0000 UTC m=+1341.085202801" watchObservedRunningTime="2026-09-11 18:37:58.856918212 +0000 UTC m=+1341.892134881"
Sep 11 18:37:59.026357 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:59.026314    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:37:59.027177 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:37:59.027158    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:38:05.700816 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.700734    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"]
Sep 11 18:38:05.704534 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.704517    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:05.707319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.707292    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-238bae-dockercfg-pbg4p\""
Sep 11 18:38:05.707428 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.707324    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-238bae\""
Sep 11 18:38:05.719797 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.719771    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"]
Sep 11 18:38:05.870066 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.870026    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786\" (UID: \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:05.870231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.870125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmsbq\" (UniqueName: \"kubernetes.io/projected/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-kube-api-access-zmsbq\") pod \"payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786\" (UID: \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:05.971491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.971407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786\" (UID: \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:05.971491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.971473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmsbq\" (UniqueName: \"kubernetes.io/projected/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-kube-api-access-zmsbq\") pod \"payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786\" (UID: \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:05.972172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.972150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786\" (UID: \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:05.980605 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:05.980586    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmsbq\" (UniqueName: \"kubernetes.io/projected/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-kube-api-access-zmsbq\") pod \"payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786\" (UID: \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:06.015793 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:06.015761    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:06.302035 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:06.302006    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"]
Sep 11 18:38:06.304604 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:38:06.304577    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e21afd3_5122_4cf0_9f22_1a2ae1b03336.slice/crio-2e14317a339e46c8a66a1d5a475b4be40c0f63bc91b0c6158f0b992b07f0cded WatchSource:0}: Error finding container 2e14317a339e46c8a66a1d5a475b4be40c0f63bc91b0c6158f0b992b07f0cded: Status 404 returned error can't find the container with id 2e14317a339e46c8a66a1d5a475b4be40c0f63bc91b0c6158f0b992b07f0cded
Sep 11 18:38:07.073552 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:07.073516    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786" event={"ID":"2e21afd3-5122-4cf0-9f22-1a2ae1b03336","Type":"ContainerStarted","Data":"3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d"}
Sep 11 18:38:07.074010 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:07.073579    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786" event={"ID":"2e21afd3-5122-4cf0-9f22-1a2ae1b03336","Type":"ContainerStarted","Data":"2e14317a339e46c8a66a1d5a475b4be40c0f63bc91b0c6158f0b992b07f0cded"}
Sep 11 18:38:07.074010 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:07.073609    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:07.096289 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:07.096228    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786" podStartSLOduration=2.096208635 podStartE2EDuration="2.096208635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:38:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:38:07.095148826 +0000 UTC m=+1350.130365497" watchObservedRunningTime="2026-09-11 18:38:07.096208635 +0000 UTC m=+1350.131425306"
Sep 11 18:38:18.081460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:18.081425    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:38:48.904415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:48.904378    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"]
Sep 11 18:38:48.909956 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:48.909933    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:38:48.912755 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:48.912729    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-699caf96\""
Sep 11 18:38:48.912755 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:48.912750    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-699caf96-dockercfg-mv794\""
Sep 11 18:38:48.925233 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:48.925200    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"]
Sep 11 18:38:49.007648 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:49.007599    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/747a6523-2d9e-40ed-93f2-faa422788dcc-plugins-config-volume\") pod \"payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm\" (UID: \"747a6523-2d9e-40ed-93f2-faa422788dcc\") " pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:38:49.007863 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:49.007699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtjcq\" (UniqueName: \"kubernetes.io/projected/747a6523-2d9e-40ed-93f2-faa422788dcc-kube-api-access-vtjcq\") pod \"payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm\" (UID: \"747a6523-2d9e-40ed-93f2-faa422788dcc\") " pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:38:49.109174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:49.109133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/747a6523-2d9e-40ed-93f2-faa422788dcc-plugins-config-volume\") pod \"payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm\" (UID: \"747a6523-2d9e-40ed-93f2-faa422788dcc\") " pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:38:49.109174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:49.109190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtjcq\" (UniqueName: \"kubernetes.io/projected/747a6523-2d9e-40ed-93f2-faa422788dcc-kube-api-access-vtjcq\") pod \"payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm\" (UID: \"747a6523-2d9e-40ed-93f2-faa422788dcc\") " pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:38:49.109934 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:49.109913    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/747a6523-2d9e-40ed-93f2-faa422788dcc-plugins-config-volume\") pod \"payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm\" (UID: \"747a6523-2d9e-40ed-93f2-faa422788dcc\") " pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:38:49.117734 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:49.117702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtjcq\" (UniqueName: \"kubernetes.io/projected/747a6523-2d9e-40ed-93f2-faa422788dcc-kube-api-access-vtjcq\") pod \"payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm\" (UID: \"747a6523-2d9e-40ed-93f2-faa422788dcc\") " pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:38:49.220999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:49.220954    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:38:49.503800 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:49.503775    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"]
Sep 11 18:38:49.505496 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:38:49.505463    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod747a6523_2d9e_40ed_93f2_faa422788dcc.slice/crio-242c4e84aab6f4fab02102d318984f6082e30f86c92c85a9e171b02df79951aa WatchSource:0}: Error finding container 242c4e84aab6f4fab02102d318984f6082e30f86c92c85a9e171b02df79951aa: Status 404 returned error can't find the container with id 242c4e84aab6f4fab02102d318984f6082e30f86c92c85a9e171b02df79951aa
Sep 11 18:38:50.328625 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:50.328581    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm" event={"ID":"747a6523-2d9e-40ed-93f2-faa422788dcc","Type":"ContainerStarted","Data":"5f2265d5a19225791c9890b1b20fe54ebe216e1fc044c9155c57ba126f0f7232"}
Sep 11 18:38:50.329088 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:50.328668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm" event={"ID":"747a6523-2d9e-40ed-93f2-faa422788dcc","Type":"ContainerStarted","Data":"242c4e84aab6f4fab02102d318984f6082e30f86c92c85a9e171b02df79951aa"}
Sep 11 18:38:50.329088 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:50.328727    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:38:54.719378 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:54.719322    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm" podStartSLOduration=6.719305827 podStartE2EDuration="6.719305827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:38:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:38:50.361944035 +0000 UTC m=+1393.397160705" watchObservedRunningTime="2026-09-11 18:38:54.719305827 +0000 UTC m=+1397.754522496"
Sep 11 18:38:54.720753 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:54.720732    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"]
Sep 11 18:38:54.724333 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:54.724318    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"
Sep 11 18:38:54.726826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:54.726804    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:38:54.726826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:54.726817    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:38:54.733547 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:54.733488    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"]
Sep 11 18:38:54.865937 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:54.865898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9cd671-6808-437b-8893-b3a805107ab0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx\" (UID: \"fd9cd671-6808-437b-8893-b3a805107ab0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"
Sep 11 18:38:54.966983 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:54.966947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9cd671-6808-437b-8893-b3a805107ab0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx\" (UID: \"fd9cd671-6808-437b-8893-b3a805107ab0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"
Sep 11 18:38:54.967579 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:54.967558    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9cd671-6808-437b-8893-b3a805107ab0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx\" (UID: \"fd9cd671-6808-437b-8893-b3a805107ab0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"
Sep 11 18:38:55.036229 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:55.036144    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"
Sep 11 18:38:55.322765 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:55.322738    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"]
Sep 11 18:38:55.324916 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:38:55.324890    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9cd671_6808_437b_8893_b3a805107ab0.slice/crio-f1c9765d276cf505b951acd795411623694171b5fecb1cf1a6e0110d51a2bed5 WatchSource:0}: Error finding container f1c9765d276cf505b951acd795411623694171b5fecb1cf1a6e0110d51a2bed5: Status 404 returned error can't find the container with id f1c9765d276cf505b951acd795411623694171b5fecb1cf1a6e0110d51a2bed5
Sep 11 18:38:55.362312 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:55.362253    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx" event={"ID":"fd9cd671-6808-437b-8893-b3a805107ab0","Type":"ContainerStarted","Data":"f1c9765d276cf505b951acd795411623694171b5fecb1cf1a6e0110d51a2bed5"}
Sep 11 18:38:56.370385 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:56.370294    2578 generic.go:356] "Generic (PLEG): container finished" podID="fd9cd671-6808-437b-8893-b3a805107ab0" containerID="ad2856119db5240434fbba42ff0b8f60c808504e86799ed6ee2001569d21ac79" exitCode=0
Sep 11 18:38:56.370788 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:56.370378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx" event={"ID":"fd9cd671-6808-437b-8893-b3a805107ab0","Type":"ContainerDied","Data":"ad2856119db5240434fbba42ff0b8f60c808504e86799ed6ee2001569d21ac79"}
Sep 11 18:38:57.513685 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:57.513648    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"
Sep 11 18:38:57.597845 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:57.597806    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9cd671-6808-437b-8893-b3a805107ab0-maas-api-service-ca\") pod \"fd9cd671-6808-437b-8893-b3a805107ab0\" (UID: \"fd9cd671-6808-437b-8893-b3a805107ab0\") "
Sep 11 18:38:57.598147 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:57.598123    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9cd671-6808-437b-8893-b3a805107ab0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd9cd671-6808-437b-8893-b3a805107ab0" (UID: "fd9cd671-6808-437b-8893-b3a805107ab0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:38:57.598279 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:57.598259    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd9cd671-6808-437b-8893-b3a805107ab0-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:38:58.383973 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:58.383938    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"
Sep 11 18:38:58.384157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:58.383980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx" event={"ID":"fd9cd671-6808-437b-8893-b3a805107ab0","Type":"ContainerDied","Data":"f1c9765d276cf505b951acd795411623694171b5fecb1cf1a6e0110d51a2bed5"}
Sep 11 18:38:58.384157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:38:58.384025    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1c9765d276cf505b951acd795411623694171b5fecb1cf1a6e0110d51a2bed5"
Sep 11 18:39:01.341134 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:01.341102    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:39:07.537937 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:07.537895    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"]
Sep 11 18:39:07.834134 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:07.834048    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"]
Sep 11 18:39:07.838373 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:07.838350    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d5a75-cscwx"]
Sep 11 18:39:09.410562 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:09.410525    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd9cd671-6808-437b-8893-b3a805107ab0" path="/var/lib/kubelet/pods/fd9cd671-6808-437b-8893-b3a805107ab0/volumes"
Sep 11 18:39:22.559038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.558973    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6" podUID="02230d2f-ccad-496b-8e95-387d1cef2548" containerName="main" containerID="cri-o://ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e" gracePeriod=45
Sep 11 18:39:22.826635 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.826556    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:39:22.954030 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954001    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-home\") pod \"02230d2f-ccad-496b-8e95-387d1cef2548\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") "
Sep 11 18:39:22.954207 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954070    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-tmp-dir\") pod \"02230d2f-ccad-496b-8e95-387d1cef2548\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") "
Sep 11 18:39:22.954207 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954101    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-model-cache\") pod \"02230d2f-ccad-496b-8e95-387d1cef2548\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") "
Sep 11 18:39:22.954207 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954142    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02230d2f-ccad-496b-8e95-387d1cef2548-tls-certs\") pod \"02230d2f-ccad-496b-8e95-387d1cef2548\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") "
Sep 11 18:39:22.954360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954227    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-dshm\") pod \"02230d2f-ccad-496b-8e95-387d1cef2548\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") "
Sep 11 18:39:22.954360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954259    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jf8z\" (UniqueName: \"kubernetes.io/projected/02230d2f-ccad-496b-8e95-387d1cef2548-kube-api-access-6jf8z\") pod \"02230d2f-ccad-496b-8e95-387d1cef2548\" (UID: \"02230d2f-ccad-496b-8e95-387d1cef2548\") "
Sep 11 18:39:22.954360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954296    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-home" (OuterVolumeSpecName: "home") pod "02230d2f-ccad-496b-8e95-387d1cef2548" (UID: "02230d2f-ccad-496b-8e95-387d1cef2548"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:22.954525 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954374    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "02230d2f-ccad-496b-8e95-387d1cef2548" (UID: "02230d2f-ccad-496b-8e95-387d1cef2548"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:22.954525 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954401    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-model-cache" (OuterVolumeSpecName: "model-cache") pod "02230d2f-ccad-496b-8e95-387d1cef2548" (UID: "02230d2f-ccad-496b-8e95-387d1cef2548"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:22.954643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954625    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-home\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:22.954697 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954645    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-tmp-dir\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:22.954697 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.954656    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-model-cache\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:22.956540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.956517    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02230d2f-ccad-496b-8e95-387d1cef2548-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "02230d2f-ccad-496b-8e95-387d1cef2548" (UID: "02230d2f-ccad-496b-8e95-387d1cef2548"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:39:22.956622 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.956584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-dshm" (OuterVolumeSpecName: "dshm") pod "02230d2f-ccad-496b-8e95-387d1cef2548" (UID: "02230d2f-ccad-496b-8e95-387d1cef2548"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:22.956622 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:22.956589    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02230d2f-ccad-496b-8e95-387d1cef2548-kube-api-access-6jf8z" (OuterVolumeSpecName: "kube-api-access-6jf8z") pod "02230d2f-ccad-496b-8e95-387d1cef2548" (UID: "02230d2f-ccad-496b-8e95-387d1cef2548"). InnerVolumeSpecName "kube-api-access-6jf8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:39:23.055266 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.055221    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02230d2f-ccad-496b-8e95-387d1cef2548-tls-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:23.055266 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.055256    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02230d2f-ccad-496b-8e95-387d1cef2548-dshm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:23.055266 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.055271    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jf8z\" (UniqueName: \"kubernetes.io/projected/02230d2f-ccad-496b-8e95-387d1cef2548-kube-api-access-6jf8z\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:23.538743 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.538706    2578 generic.go:356] "Generic (PLEG): container finished" podID="02230d2f-ccad-496b-8e95-387d1cef2548" containerID="ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e" exitCode=0
Sep 11 18:39:23.538882 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.538771    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"
Sep 11 18:39:23.538882 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.538788    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6" event={"ID":"02230d2f-ccad-496b-8e95-387d1cef2548","Type":"ContainerDied","Data":"ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e"}
Sep 11 18:39:23.538882 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.538831    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6" event={"ID":"02230d2f-ccad-496b-8e95-387d1cef2548","Type":"ContainerDied","Data":"1d2812e9fcb92b8bef9968a2f1a324d3e3f67c4b932806cd74f7011af35fb2b1"}
Sep 11 18:39:23.538882 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.538848    2578 scope.go:117] "RemoveContainer" containerID="ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e"
Sep 11 18:39:23.547755 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.547735    2578 scope.go:117] "RemoveContainer" containerID="ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e"
Sep 11 18:39:23.548070 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:39:23.548049    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e\": container with ID starting with ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e not found: ID does not exist" containerID="ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e"
Sep 11 18:39:23.548143 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.548081    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e"} err="failed to get container status \"ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e\": rpc error: code = NotFound desc = could not find container \"ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e\": container with ID starting with ce961afad8cccb39f0054790b82e2589720c4f397835f7f42507b3d43fc0948e not found: ID does not exist"
Sep 11 18:39:23.559847 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.559820    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"]
Sep 11 18:39:23.565925 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:23.565901    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-4d5a75/test-model-4d5a75-kserve-8577cbcc69-bchb6"]
Sep 11 18:39:25.411139 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:25.411108    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02230d2f-ccad-496b-8e95-387d1cef2548" path="/var/lib/kubelet/pods/02230d2f-ccad-496b-8e95-387d1cef2548/volumes"
Sep 11 18:39:37.561541 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:37.561510    2578 scope.go:117] "RemoveContainer" containerID="076cac05e38693c032433a8aed585ff395cbea2d2fc269d2faad37f4bfe5ac0b"
Sep 11 18:39:37.570222 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:37.570198    2578 scope.go:117] "RemoveContainer" containerID="4060da1820382bff6378dfd3cdc206843f7afc726dc373484081a2364140b85b"
Sep 11 18:39:50.565721 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.565662    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"]
Sep 11 18:39:50.566472 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.565966    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy" containerID="cri-o://d5d961fea269ef5f1258dd2c80a023a385e1b9043a643723e710a42f7cd0d205" gracePeriod=30
Sep 11 18:39:50.969220 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.969179    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"]
Sep 11 18:39:50.969627 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.969614    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd9cd671-6808-437b-8893-b3a805107ab0" containerName="revoke-keys"
Sep 11 18:39:50.969627 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.969628    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd9cd671-6808-437b-8893-b3a805107ab0" containerName="revoke-keys"
Sep 11 18:39:50.969750 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.969644    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02230d2f-ccad-496b-8e95-387d1cef2548" containerName="main"
Sep 11 18:39:50.969750 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.969649    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="02230d2f-ccad-496b-8e95-387d1cef2548" containerName="main"
Sep 11 18:39:50.969750 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.969733    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="02230d2f-ccad-496b-8e95-387d1cef2548" containerName="main"
Sep 11 18:39:50.969750 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.969742    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd9cd671-6808-437b-8893-b3a805107ab0" containerName="revoke-keys"
Sep 11 18:39:50.973019 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.973001    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"
Sep 11 18:39:50.975458 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.975424    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:39:50.975613 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.975521    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:39:50.986225 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:50.986194    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"]
Sep 11 18:39:51.044745 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:51.044705    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307fc8c9-df84-464a-8528-5074f2697fa8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt\" (UID: \"307fc8c9-df84-464a-8528-5074f2697fa8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"
Sep 11 18:39:51.145884 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:51.145847    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307fc8c9-df84-464a-8528-5074f2697fa8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt\" (UID: \"307fc8c9-df84-464a-8528-5074f2697fa8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"
Sep 11 18:39:51.146499 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:51.146479    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307fc8c9-df84-464a-8528-5074f2697fa8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt\" (UID: \"307fc8c9-df84-464a-8528-5074f2697fa8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"
Sep 11 18:39:51.283921 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:51.283821    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"
Sep 11 18:39:51.553927 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:51.553896    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"]
Sep 11 18:39:51.556085 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:39:51.556058    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307fc8c9_df84_464a_8528_5074f2697fa8.slice/crio-1c2ff6d194c1087c3e9b63a5e623597fccc81496dcf595b187bb5eac346b1e91 WatchSource:0}: Error finding container 1c2ff6d194c1087c3e9b63a5e623597fccc81496dcf595b187bb5eac346b1e91: Status 404 returned error can't find the container with id 1c2ff6d194c1087c3e9b63a5e623597fccc81496dcf595b187bb5eac346b1e91
Sep 11 18:39:51.708256 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:51.708221    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt" event={"ID":"307fc8c9-df84-464a-8528-5074f2697fa8","Type":"ContainerStarted","Data":"1c2ff6d194c1087c3e9b63a5e623597fccc81496dcf595b187bb5eac346b1e91"}
Sep 11 18:39:52.715554 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:52.715519    2578 generic.go:356] "Generic (PLEG): container finished" podID="307fc8c9-df84-464a-8528-5074f2697fa8" containerID="63ff34ce224d2bf57b578e516f91bf1b13074c5e5f9f8b3131e96282f868c6b5" exitCode=0
Sep 11 18:39:52.716005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:52.715604    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt" event={"ID":"307fc8c9-df84-464a-8528-5074f2697fa8","Type":"ContainerDied","Data":"63ff34ce224d2bf57b578e516f91bf1b13074c5e5f9f8b3131e96282f868c6b5"}
Sep 11 18:39:53.859745 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:53.859714    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"
Sep 11 18:39:53.976530 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:53.976434    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307fc8c9-df84-464a-8528-5074f2697fa8-maas-api-service-ca\") pod \"307fc8c9-df84-464a-8528-5074f2697fa8\" (UID: \"307fc8c9-df84-464a-8528-5074f2697fa8\") "
Sep 11 18:39:53.976887 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:53.976858    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/307fc8c9-df84-464a-8528-5074f2697fa8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "307fc8c9-df84-464a-8528-5074f2697fa8" (UID: "307fc8c9-df84-464a-8528-5074f2697fa8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:39:54.077998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:54.077956    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307fc8c9-df84-464a-8528-5074f2697fa8-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:54.728600 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:54.728571    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"
Sep 11 18:39:54.728809 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:54.728565    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt" event={"ID":"307fc8c9-df84-464a-8528-5074f2697fa8","Type":"ContainerDied","Data":"1c2ff6d194c1087c3e9b63a5e623597fccc81496dcf595b187bb5eac346b1e91"}
Sep 11 18:39:54.728809 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:54.728761    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c2ff6d194c1087c3e9b63a5e623597fccc81496dcf595b187bb5eac346b1e91"
Sep 11 18:39:55.737621 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.737591    2578 generic.go:356] "Generic (PLEG): container finished" podID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerID="d5d961fea269ef5f1258dd2c80a023a385e1b9043a643723e710a42f7cd0d205" exitCode=0
Sep 11 18:39:55.738038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.737629    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" event={"ID":"bd6fc16f-db5c-4df4-8d13-5911feba2ee2","Type":"ContainerDied","Data":"d5d961fea269ef5f1258dd2c80a023a385e1b9043a643723e710a42f7cd0d205"}
Sep 11 18:39:55.738038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.737694    2578 scope.go:117] "RemoveContainer" containerID="1acf7b775eb9fec9290d2d22b6404da69510494684f8e7ddd7af40cd932e2815"
Sep 11 18:39:55.843429 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.843359    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:39:55.998604 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.998569    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-data\") pod \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") "
Sep 11 18:39:55.998830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.998614    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-podinfo\") pod \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") "
Sep 11 18:39:55.998830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.998635    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-certs\") pod \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") "
Sep 11 18:39:55.998830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.998668    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-socket\") pod \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") "
Sep 11 18:39:55.998830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.998736    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tm6w6\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-kube-api-access-tm6w6\") pod \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") "
Sep 11 18:39:55.999044 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.998832    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istiod-ca-cert\") pod \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") "
Sep 11 18:39:55.999044 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.998888    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-envoy\") pod \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") "
Sep 11 18:39:55.999044 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.998916    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-credential-socket\") pod \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") "
Sep 11 18:39:55.999044 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999006    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd6fc16f-db5c-4df4-8d13-5911feba2ee2" (UID: "bd6fc16f-db5c-4df4-8d13-5911feba2ee2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:55.999044 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999035    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd6fc16f-db5c-4df4-8d13-5911feba2ee2" (UID: "bd6fc16f-db5c-4df4-8d13-5911feba2ee2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:55.999319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999013    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd6fc16f-db5c-4df4-8d13-5911feba2ee2" (UID: "bd6fc16f-db5c-4df4-8d13-5911feba2ee2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:55.999319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999069    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-token\") pod \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\" (UID: \"bd6fc16f-db5c-4df4-8d13-5911feba2ee2\") "
Sep 11 18:39:55.999319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999168    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd6fc16f-db5c-4df4-8d13-5911feba2ee2" (UID: "bd6fc16f-db5c-4df4-8d13-5911feba2ee2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:55.999319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999248    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd6fc16f-db5c-4df4-8d13-5911feba2ee2" (UID: "bd6fc16f-db5c-4df4-8d13-5911feba2ee2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:39:55.999535 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999507    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:55.999535 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999524    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:55.999535 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999535    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:55.999710 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999546    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:55.999710 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:55.999558    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:56.001160 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.001135    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd6fc16f-db5c-4df4-8d13-5911feba2ee2" (UID: "bd6fc16f-db5c-4df4-8d13-5911feba2ee2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:39:56.001283 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.001154    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-kube-api-access-tm6w6" (OuterVolumeSpecName: "kube-api-access-tm6w6") pod "bd6fc16f-db5c-4df4-8d13-5911feba2ee2" (UID: "bd6fc16f-db5c-4df4-8d13-5911feba2ee2"). InnerVolumeSpecName "kube-api-access-tm6w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:39:56.001283 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.001206    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd6fc16f-db5c-4df4-8d13-5911feba2ee2" (UID: "bd6fc16f-db5c-4df4-8d13-5911feba2ee2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:39:56.001283 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.001220    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd6fc16f-db5c-4df4-8d13-5911feba2ee2" (UID: "bd6fc16f-db5c-4df4-8d13-5911feba2ee2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:56.101056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.100970    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm6w6\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-kube-api-access-tm6w6\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:56.101056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.101000    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:56.101056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.101011    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:56.101056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.101020    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd6fc16f-db5c-4df4-8d13-5911feba2ee2-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:39:56.745037 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.745009    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"
Sep 11 18:39:56.745496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.745009    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp" event={"ID":"bd6fc16f-db5c-4df4-8d13-5911feba2ee2","Type":"ContainerDied","Data":"826310f913a80f40f108c1b4846a7c76b805039b2f181d7bb0d6653fa05a4098"}
Sep 11 18:39:56.745496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.745142    2578 scope.go:117] "RemoveContainer" containerID="d5d961fea269ef5f1258dd2c80a023a385e1b9043a643723e710a42f7cd0d205"
Sep 11 18:39:56.771873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.771838    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"]
Sep 11 18:39:56.777579 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:56.777551    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4d5a75-openshift-default-b858f6c66-rn7kp"]
Sep 11 18:39:57.411458 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:39:57.411423    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" path="/var/lib/kubelet/pods/bd6fc16f-db5c-4df4-8d13-5911feba2ee2/volumes"
Sep 11 18:40:01.603597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.603564    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"]
Sep 11 18:40:01.604128 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.603833    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln" podUID="fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8" containerName="payload-pre-processing" containerID="cri-o://c245b019e27a98f982e01a252f06610101ba77c2e0409c2006eb3473a16258fd" gracePeriod=30
Sep 11 18:40:01.778807 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.778770    2578 generic.go:356] "Generic (PLEG): container finished" podID="fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8" containerID="c245b019e27a98f982e01a252f06610101ba77c2e0409c2006eb3473a16258fd" exitCode=0
Sep 11 18:40:01.778974 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.778849    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln" event={"ID":"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8","Type":"ContainerDied","Data":"c245b019e27a98f982e01a252f06610101ba77c2e0409c2006eb3473a16258fd"}
Sep 11 18:40:01.811735 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.811669    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"]
Sep 11 18:40:01.815555 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.815522    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e8f201-pxhvt"]
Sep 11 18:40:01.854637 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.854576    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:40:01.956147 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.956112    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8fdd\" (UniqueName: \"kubernetes.io/projected/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-kube-api-access-t8fdd\") pod \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\" (UID: \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\") "
Sep 11 18:40:01.956147 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.956151    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-plugins-config-volume\") pod \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\" (UID: \"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8\") "
Sep 11 18:40:01.956554 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.956528    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8" (UID: "fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:40:01.958194 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:01.958168    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-kube-api-access-t8fdd" (OuterVolumeSpecName: "kube-api-access-t8fdd") pod "fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8" (UID: "fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8"). InnerVolumeSpecName "kube-api-access-t8fdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:40:02.057102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:02.057067    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8fdd\" (UniqueName: \"kubernetes.io/projected/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-kube-api-access-t8fdd\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:02.057102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:02.057095    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:02.785187 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:02.785158    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"
Sep 11 18:40:02.785635 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:02.785192    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln" event={"ID":"fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8","Type":"ContainerDied","Data":"0351b0f644a06ab6c09ac07543c13242f139ddc9c0dab1d3f9cb55bdd30c770a"}
Sep 11 18:40:02.785635 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:02.785246    2578 scope.go:117] "RemoveContainer" containerID="c245b019e27a98f982e01a252f06610101ba77c2e0409c2006eb3473a16258fd"
Sep 11 18:40:02.813079 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:02.813040    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"]
Sep 11 18:40:02.816162 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:02.816141    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8f201-6798855ccd-4lpln"]
Sep 11 18:40:03.410590 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:03.410555    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="307fc8c9-df84-464a-8528-5074f2697fa8" path="/var/lib/kubelet/pods/307fc8c9-df84-464a-8528-5074f2697fa8/volumes"
Sep 11 18:40:03.410990 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:03.410976    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8" path="/var/lib/kubelet/pods/fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8/volumes"
Sep 11 18:40:07.549643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:07.549605    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"]
Sep 11 18:40:07.550174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:07.549846    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7" podUID="757ddd53-e60d-4a21-9676-702abca84956" containerName="istio-proxy" containerID="cri-o://aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a" gracePeriod=30
Sep 11 18:40:12.804546 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.804522    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:40:12.841471 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.841389    2578 generic.go:356] "Generic (PLEG): container finished" podID="757ddd53-e60d-4a21-9676-702abca84956" containerID="aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a" exitCode=0
Sep 11 18:40:12.841471 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.841450    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"
Sep 11 18:40:12.841471 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.841463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7" event={"ID":"757ddd53-e60d-4a21-9676-702abca84956","Type":"ContainerDied","Data":"aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a"}
Sep 11 18:40:12.841727 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.841503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7" event={"ID":"757ddd53-e60d-4a21-9676-702abca84956","Type":"ContainerDied","Data":"90961f7796466d0791389b417dcd94f59f6f906deffbe39ec2a342b41c982e7b"}
Sep 11 18:40:12.841727 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.841525    2578 scope.go:117] "RemoveContainer" containerID="aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a"
Sep 11 18:40:12.851096 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.851067    2578 scope.go:117] "RemoveContainer" containerID="aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a"
Sep 11 18:40:12.851385 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:40:12.851363    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a\": container with ID starting with aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a not found: ID does not exist" containerID="aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a"
Sep 11 18:40:12.851475 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.851393    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a"} err="failed to get container status \"aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a\": rpc error: code = NotFound desc = could not find container \"aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a\": container with ID starting with aed29374d1ad11328ec9d1f7a81e56fcfded8e135112df3a0e2f8609f40caa0a not found: ID does not exist"
Sep 11 18:40:12.861734 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.861716    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-istio-token\") pod \"757ddd53-e60d-4a21-9676-702abca84956\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") "
Sep 11 18:40:12.861818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.861754    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-envoy\") pod \"757ddd53-e60d-4a21-9676-702abca84956\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") "
Sep 11 18:40:12.861818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.861775    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpqvw\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-kube-api-access-gpqvw\") pod \"757ddd53-e60d-4a21-9676-702abca84956\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") "
Sep 11 18:40:12.861818 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.861813    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/757ddd53-e60d-4a21-9676-702abca84956-istiod-ca-cert\") pod \"757ddd53-e60d-4a21-9676-702abca84956\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") "
Sep 11 18:40:12.861984 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.861857    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-certs\") pod \"757ddd53-e60d-4a21-9676-702abca84956\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") "
Sep 11 18:40:12.861984 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.861887    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/757ddd53-e60d-4a21-9676-702abca84956-istio-podinfo\") pod \"757ddd53-e60d-4a21-9676-702abca84956\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") "
Sep 11 18:40:12.861984 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.861962    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-data\") pod \"757ddd53-e60d-4a21-9676-702abca84956\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") "
Sep 11 18:40:12.862134 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.861988    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-socket\") pod \"757ddd53-e60d-4a21-9676-702abca84956\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") "
Sep 11 18:40:12.862134 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.862046    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-credential-socket\") pod \"757ddd53-e60d-4a21-9676-702abca84956\" (UID: \"757ddd53-e60d-4a21-9676-702abca84956\") "
Sep 11 18:40:12.862281 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.862247    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-data" (OuterVolumeSpecName: "istio-data") pod "757ddd53-e60d-4a21-9676-702abca84956" (UID: "757ddd53-e60d-4a21-9676-702abca84956"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:40:12.862363 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.862332    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "757ddd53-e60d-4a21-9676-702abca84956" (UID: "757ddd53-e60d-4a21-9676-702abca84956"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:40:12.862424 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.862358    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/757ddd53-e60d-4a21-9676-702abca84956-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "757ddd53-e60d-4a21-9676-702abca84956" (UID: "757ddd53-e60d-4a21-9676-702abca84956"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:40:12.862507 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.862486    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "757ddd53-e60d-4a21-9676-702abca84956" (UID: "757ddd53-e60d-4a21-9676-702abca84956"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:40:12.862566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.862505    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/757ddd53-e60d-4a21-9676-702abca84956-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:12.862566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.862526    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:12.862566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.862522    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "757ddd53-e60d-4a21-9676-702abca84956" (UID: "757ddd53-e60d-4a21-9676-702abca84956"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:40:12.862566 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.862539    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:12.864126 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.864101    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "757ddd53-e60d-4a21-9676-702abca84956" (UID: "757ddd53-e60d-4a21-9676-702abca84956"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:40:12.864126 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.864121    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-istio-token" (OuterVolumeSpecName: "istio-token") pod "757ddd53-e60d-4a21-9676-702abca84956" (UID: "757ddd53-e60d-4a21-9676-702abca84956"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:40:12.864273 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.864246    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/757ddd53-e60d-4a21-9676-702abca84956-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "757ddd53-e60d-4a21-9676-702abca84956" (UID: "757ddd53-e60d-4a21-9676-702abca84956"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:40:12.864315 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.864253    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-kube-api-access-gpqvw" (OuterVolumeSpecName: "kube-api-access-gpqvw") pod "757ddd53-e60d-4a21-9676-702abca84956" (UID: "757ddd53-e60d-4a21-9676-702abca84956"). InnerVolumeSpecName "kube-api-access-gpqvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:40:12.963557 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.963520    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:12.963557 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.963550    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:12.963557 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.963560    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:12.963867 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.963568    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpqvw\" (UniqueName: \"kubernetes.io/projected/757ddd53-e60d-4a21-9676-702abca84956-kube-api-access-gpqvw\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:12.963867 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.963578    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/757ddd53-e60d-4a21-9676-702abca84956-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:12.963867 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:12.963586    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/757ddd53-e60d-4a21-9676-702abca84956-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:13.167069 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:13.167034    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"]
Sep 11 18:40:13.171242 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:13.171216    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-3b7488-openshift-default-5dcd49db56-m27w7"]
Sep 11 18:40:13.410689 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:13.410656    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="757ddd53-e60d-4a21-9676-702abca84956" path="/var/lib/kubelet/pods/757ddd53-e60d-4a21-9676-702abca84956/volumes"
Sep 11 18:40:21.350012 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.349977    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"]
Sep 11 18:40:21.350619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350597    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="307fc8c9-df84-464a-8528-5074f2697fa8" containerName="revoke-keys"
Sep 11 18:40:21.350737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350622    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="307fc8c9-df84-464a-8528-5074f2697fa8" containerName="revoke-keys"
Sep 11 18:40:21.350737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350643    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757ddd53-e60d-4a21-9676-702abca84956" containerName="istio-proxy"
Sep 11 18:40:21.350737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350652    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="757ddd53-e60d-4a21-9676-702abca84956" containerName="istio-proxy"
Sep 11 18:40:21.350737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350671    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy"
Sep 11 18:40:21.350737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350723    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy"
Sep 11 18:40:21.350737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350739    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy"
Sep 11 18:40:21.351036 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350748    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy"
Sep 11 18:40:21.351036 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350765    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8" containerName="payload-pre-processing"
Sep 11 18:40:21.351036 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350773    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8" containerName="payload-pre-processing"
Sep 11 18:40:21.351036 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350889    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="307fc8c9-df84-464a-8528-5074f2697fa8" containerName="revoke-keys"
Sep 11 18:40:21.351036 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350902    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy"
Sep 11 18:40:21.351036 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350914    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="757ddd53-e60d-4a21-9676-702abca84956" containerName="istio-proxy"
Sep 11 18:40:21.351036 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350922    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd6fc16f-db5c-4df4-8d13-5911feba2ee2" containerName="istio-proxy"
Sep 11 18:40:21.351036 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.350931    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdc293e3-7ffc-4966-8e4e-8c7a82fa84e8" containerName="payload-pre-processing"
Sep 11 18:40:21.355630 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.355608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.358887 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.358854    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-699caf96\"/\"kube-root-ca.crt\""
Sep 11 18:40:21.359068 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.359053    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-699caf96\"/\"default-dockercfg-jm5bl\""
Sep 11 18:40:21.359904 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.359884    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-699caf96\"/\"e2e-lifecycle-model-699caf96-kserve-self-signed-certs\""
Sep 11 18:40:21.359997 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.359891    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-699caf96\"/\"openshift-service-ca.crt\""
Sep 11 18:40:21.376846 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.376817    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"]
Sep 11 18:40:21.444423 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.444394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-model-cache\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.444423 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.444428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-home\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.444709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.444554    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-tmp-dir\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.444709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.444624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-dshm\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.444709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.444671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm9pm\" (UniqueName: \"kubernetes.io/projected/5208bf9d-1889-4892-81c7-5ccc25137746-kube-api-access-qm9pm\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.444850 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.444717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5208bf9d-1889-4892-81c7-5ccc25137746-tls-certs\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.545794 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.545745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-dshm\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.545989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.545818    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm9pm\" (UniqueName: \"kubernetes.io/projected/5208bf9d-1889-4892-81c7-5ccc25137746-kube-api-access-qm9pm\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.545989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.545841    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5208bf9d-1889-4892-81c7-5ccc25137746-tls-certs\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.545989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.545882    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-model-cache\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.545989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.545902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-home\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.545989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.545967    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-tmp-dir\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.546259 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.546238    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-model-cache\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.546310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.546293    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-tmp-dir\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.546350 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.546318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-home\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.548136 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.548111    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-dshm\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.548288 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.548271    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5208bf9d-1889-4892-81c7-5ccc25137746-tls-certs\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.555382 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.555356    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm9pm\" (UniqueName: \"kubernetes.io/projected/5208bf9d-1889-4892-81c7-5ccc25137746-kube-api-access-qm9pm\") pod \"e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") " pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.665275 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.665235    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:21.946316 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:21.946233    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"]
Sep 11 18:40:21.949233 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:40:21.949204    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5208bf9d_1889_4892_81c7_5ccc25137746.slice/crio-0972c45b84ceefa82eb86d51c774aed5f2d2dc3b662f5a1e8abc28831b34f15f WatchSource:0}: Error finding container 0972c45b84ceefa82eb86d51c774aed5f2d2dc3b662f5a1e8abc28831b34f15f: Status 404 returned error can't find the container with id 0972c45b84ceefa82eb86d51c774aed5f2d2dc3b662f5a1e8abc28831b34f15f
Sep 11 18:40:22.899054 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:22.899014    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx" event={"ID":"5208bf9d-1889-4892-81c7-5ccc25137746","Type":"ContainerStarted","Data":"90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d"}
Sep 11 18:40:22.899054 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:22.899061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx" event={"ID":"5208bf9d-1889-4892-81c7-5ccc25137746","Type":"ContainerStarted","Data":"0972c45b84ceefa82eb86d51c774aed5f2d2dc3b662f5a1e8abc28831b34f15f"}
Sep 11 18:40:22.918065 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:22.918016    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx" podStartSLOduration=1.918000702 podStartE2EDuration="1.918000702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:40:22.916728889 +0000 UTC m=+1485.951945582" watchObservedRunningTime="2026-09-11 18:40:22.918000702 +0000 UTC m=+1485.953217422"
Sep 11 18:40:31.666299 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:31.666253    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:31.666745 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:31.666319    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:31.679129 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:31.679102    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:31.958502 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:31.958409    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:40:37.482643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:37.482610    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:40:37.486835 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:37.486810    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:40:37.490170 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:37.490153    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:40:37.494295 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:37.494279    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:40:37.601031 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:37.601001    2578 scope.go:117] "RemoveContainer" containerID="dd862c909b8c36d0ef132d0681e59467c5d3e1800d680edc1b1d7330a3fd2c0e"
Sep 11 18:40:37.609814 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:37.609787    2578 scope.go:117] "RemoveContainer" containerID="5baef964c75cc0d2c3b99cbbe334e5452ea412540a35a3e2122b763f199c9b32"
Sep 11 18:40:45.528548 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.528510    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"]
Sep 11 18:40:45.532214 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.532198    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.535084 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.535059    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-d90e71-openshift-default-dockercfg-668gj\""
Sep 11 18:40:45.561469 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.561440    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"]
Sep 11 18:40:45.575889 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.575861    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-data\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.576038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.575901    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/840dc218-df17-4626-b356-b01616169418-istio-podinfo\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.576038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.575975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/840dc218-df17-4626-b356-b01616169418-istiod-ca-cert\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.576038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.575996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-credential-socket\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.576038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.576027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-envoy\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.576179 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.576066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-istio-token\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.576179 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.576099    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-certs\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.576265 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.576192    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-socket\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.576320 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.576270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78wq5\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-kube-api-access-78wq5\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677183 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-socket\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677183 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78wq5\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-kube-api-access-78wq5\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677427 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-data\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677427 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/840dc218-df17-4626-b356-b01616169418-istio-podinfo\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677427 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/840dc218-df17-4626-b356-b01616169418-istiod-ca-cert\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677427 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-credential-socket\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677427 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-envoy\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677427 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-istio-token\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677427 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677382    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-certs\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677834 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677614    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-socket\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677834 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677780    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-data\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677834 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-credential-socket\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.677834 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.677785    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-certs\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.678046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.678030    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/840dc218-df17-4626-b356-b01616169418-istiod-ca-cert\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.679650 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.679630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-envoy\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.679946 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.679927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/840dc218-df17-4626-b356-b01616169418-istio-podinfo\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.690149 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.690116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-istio-token\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.690341 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.690322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78wq5\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-kube-api-access-78wq5\") pod \"e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c\" (UID: \"840dc218-df17-4626-b356-b01616169418\") " pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:45.841990 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:45.841904    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:46.119538 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:46.119452    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"]
Sep 11 18:40:46.122860 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:40:46.122831    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod840dc218_df17_4626_b356_b01616169418.slice/crio-13fd67fac58bf0c4134529f3338b57f4ed7ad2675eaf67f340ec1133c4fa840a WatchSource:0}: Error finding container 13fd67fac58bf0c4134529f3338b57f4ed7ad2675eaf67f340ec1133c4fa840a: Status 404 returned error can't find the container with id 13fd67fac58bf0c4134529f3338b57f4ed7ad2675eaf67f340ec1133c4fa840a
Sep 11 18:40:46.125178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:46.125148    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:40:46.125254 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:46.125214    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:40:46.125254 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:46.125243    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:40:47.036654 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.036616    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" event={"ID":"840dc218-df17-4626-b356-b01616169418","Type":"ContainerStarted","Data":"34f8df5ade8cc09f2f478ffe5906b1bebede3c1b902a063ab27cd6d41795d40e"}
Sep 11 18:40:47.037076 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.036702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" event={"ID":"840dc218-df17-4626-b356-b01616169418","Type":"ContainerStarted","Data":"13fd67fac58bf0c4134529f3338b57f4ed7ad2675eaf67f340ec1133c4fa840a"}
Sep 11 18:40:47.069950 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.069902    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" podStartSLOduration=2.069887187 podStartE2EDuration="2.069887187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:40:47.066951554 +0000 UTC m=+1510.102168226" watchObservedRunningTime="2026-09-11 18:40:47.069887187 +0000 UTC m=+1510.105103858"
Sep 11 18:40:47.730271 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.730241    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"]
Sep 11 18:40:47.741023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.740991    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"
Sep 11 18:40:47.746541 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.746516    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"]
Sep 11 18:40:47.747670 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.747647    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:40:47.747829 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.747647    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:40:47.799742 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.799713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/076ff3cd-dd7d-4abb-85cf-5bb996e41189-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-047453-dbwts\" (UID: \"076ff3cd-dd7d-4abb-85cf-5bb996e41189\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"
Sep 11 18:40:47.842227 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.842200    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:47.847039 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.847016    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:47.901182 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.901152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/076ff3cd-dd7d-4abb-85cf-5bb996e41189-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-047453-dbwts\" (UID: \"076ff3cd-dd7d-4abb-85cf-5bb996e41189\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"
Sep 11 18:40:47.901796 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:47.901779    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/076ff3cd-dd7d-4abb-85cf-5bb996e41189-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-047453-dbwts\" (UID: \"076ff3cd-dd7d-4abb-85cf-5bb996e41189\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"
Sep 11 18:40:48.042650 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:48.042558    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:48.043361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:48.043342    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:40:48.054755 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:48.054734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"
Sep 11 18:40:48.339910 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:48.339878    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"]
Sep 11 18:40:48.341098 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:40:48.341075    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076ff3cd_dd7d_4abb_85cf_5bb996e41189.slice/crio-45b5a78e405c2c981062285f9889abcad9e036270265ff46bf71cba52f65cf9c WatchSource:0}: Error finding container 45b5a78e405c2c981062285f9889abcad9e036270265ff46bf71cba52f65cf9c: Status 404 returned error can't find the container with id 45b5a78e405c2c981062285f9889abcad9e036270265ff46bf71cba52f65cf9c
Sep 11 18:40:49.052644 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:49.052600    2578 generic.go:356] "Generic (PLEG): container finished" podID="076ff3cd-dd7d-4abb-85cf-5bb996e41189" containerID="b7ee72b4ac32e558a966eff0accfc0654e5deb52e461a53b5843db01baf54aa6" exitCode=0
Sep 11 18:40:49.053035 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:49.052667    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts" event={"ID":"076ff3cd-dd7d-4abb-85cf-5bb996e41189","Type":"ContainerDied","Data":"b7ee72b4ac32e558a966eff0accfc0654e5deb52e461a53b5843db01baf54aa6"}
Sep 11 18:40:49.053035 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:49.052749    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts" event={"ID":"076ff3cd-dd7d-4abb-85cf-5bb996e41189","Type":"ContainerStarted","Data":"45b5a78e405c2c981062285f9889abcad9e036270265ff46bf71cba52f65cf9c"}
Sep 11 18:40:50.195765 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:50.195737    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"
Sep 11 18:40:50.222578 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:50.222545    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/076ff3cd-dd7d-4abb-85cf-5bb996e41189-maas-api-service-ca\") pod \"076ff3cd-dd7d-4abb-85cf-5bb996e41189\" (UID: \"076ff3cd-dd7d-4abb-85cf-5bb996e41189\") "
Sep 11 18:40:50.222937 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:50.222908    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/076ff3cd-dd7d-4abb-85cf-5bb996e41189-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "076ff3cd-dd7d-4abb-85cf-5bb996e41189" (UID: "076ff3cd-dd7d-4abb-85cf-5bb996e41189"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:40:50.324150 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:50.324062    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/076ff3cd-dd7d-4abb-85cf-5bb996e41189-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:40:51.064497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:51.064462    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts" event={"ID":"076ff3cd-dd7d-4abb-85cf-5bb996e41189","Type":"ContainerDied","Data":"45b5a78e405c2c981062285f9889abcad9e036270265ff46bf71cba52f65cf9c"}
Sep 11 18:40:51.064705 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:51.064507    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"
Sep 11 18:40:51.064705 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:51.064512    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45b5a78e405c2c981062285f9889abcad9e036270265ff46bf71cba52f65cf9c"
Sep 11 18:40:52.577345 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:40:52.577310    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"]
Sep 11 18:41:03.851435 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:03.851346    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"]
Sep 11 18:41:03.851994 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:03.851628    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675" podUID="51d68738-4236-4ed2-8367-13bb27412e8f" containerName="payload-processing" containerID="cri-o://d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a" gracePeriod=30
Sep 11 18:41:03.862593 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:03.862517    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"]
Sep 11 18:41:03.862811 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:03.862783    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9" podUID="589e13de-5fc0-44e4-848c-4bf774d2fde6" containerName="payload-pre-processing" containerID="cri-o://36b1764aee17a62a0038043b09a60d80aa9417e66a15de49c7a3e87699c797c0" gracePeriod=30
Sep 11 18:41:04.051415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.051373    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"]
Sep 11 18:41:04.054005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.053975    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-047453-dbwts"]
Sep 11 18:41:04.163125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.163003    2578 generic.go:356] "Generic (PLEG): container finished" podID="589e13de-5fc0-44e4-848c-4bf774d2fde6" containerID="36b1764aee17a62a0038043b09a60d80aa9417e66a15de49c7a3e87699c797c0" exitCode=0
Sep 11 18:41:04.163611 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.163563    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9" event={"ID":"589e13de-5fc0-44e4-848c-4bf774d2fde6","Type":"ContainerDied","Data":"36b1764aee17a62a0038043b09a60d80aa9417e66a15de49c7a3e87699c797c0"}
Sep 11 18:41:04.165120 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.165040    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:41:04.166200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.165669    2578 generic.go:356] "Generic (PLEG): container finished" podID="51d68738-4236-4ed2-8367-13bb27412e8f" containerID="d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a" exitCode=0
Sep 11 18:41:04.166200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.165799    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675" event={"ID":"51d68738-4236-4ed2-8367-13bb27412e8f","Type":"ContainerDied","Data":"d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a"}
Sep 11 18:41:04.166200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.165846    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675" event={"ID":"51d68738-4236-4ed2-8367-13bb27412e8f","Type":"ContainerDied","Data":"b5307384202bba277fdc3388e47237b6640a698cb7de8ac22a18b8c5ea00f6ae"}
Sep 11 18:41:04.166200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.165875    2578 scope.go:117] "RemoveContainer" containerID="d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a"
Sep 11 18:41:04.179638 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.179607    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:41:04.181047 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.181029    2578 scope.go:117] "RemoveContainer" containerID="d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a"
Sep 11 18:41:04.181343 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:41:04.181321    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a\": container with ID starting with d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a not found: ID does not exist" containerID="d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a"
Sep 11 18:41:04.181442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.181350    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a"} err="failed to get container status \"d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a\": rpc error: code = NotFound desc = could not find container \"d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a\": container with ID starting with d0730b59b36e7523238ebeddab04c5b8f4957081b5a296218e160d2eae36500a not found: ID does not exist"
Sep 11 18:41:04.257179 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.257145    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51d68738-4236-4ed2-8367-13bb27412e8f-plugins-config-volume\") pod \"51d68738-4236-4ed2-8367-13bb27412e8f\" (UID: \"51d68738-4236-4ed2-8367-13bb27412e8f\") "
Sep 11 18:41:04.257179 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.257189    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hstrw\" (UniqueName: \"kubernetes.io/projected/589e13de-5fc0-44e4-848c-4bf774d2fde6-kube-api-access-hstrw\") pod \"589e13de-5fc0-44e4-848c-4bf774d2fde6\" (UID: \"589e13de-5fc0-44e4-848c-4bf774d2fde6\") "
Sep 11 18:41:04.257412 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.257229    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589e13de-5fc0-44e4-848c-4bf774d2fde6-plugins-config-volume\") pod \"589e13de-5fc0-44e4-848c-4bf774d2fde6\" (UID: \"589e13de-5fc0-44e4-848c-4bf774d2fde6\") "
Sep 11 18:41:04.257412 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.257271    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lxkv\" (UniqueName: \"kubernetes.io/projected/51d68738-4236-4ed2-8367-13bb27412e8f-kube-api-access-5lxkv\") pod \"51d68738-4236-4ed2-8367-13bb27412e8f\" (UID: \"51d68738-4236-4ed2-8367-13bb27412e8f\") "
Sep 11 18:41:04.257578 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.257552    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51d68738-4236-4ed2-8367-13bb27412e8f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "51d68738-4236-4ed2-8367-13bb27412e8f" (UID: "51d68738-4236-4ed2-8367-13bb27412e8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:41:04.257698 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.257652    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/589e13de-5fc0-44e4-848c-4bf774d2fde6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "589e13de-5fc0-44e4-848c-4bf774d2fde6" (UID: "589e13de-5fc0-44e4-848c-4bf774d2fde6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:41:04.259458 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.259438    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589e13de-5fc0-44e4-848c-4bf774d2fde6-kube-api-access-hstrw" (OuterVolumeSpecName: "kube-api-access-hstrw") pod "589e13de-5fc0-44e4-848c-4bf774d2fde6" (UID: "589e13de-5fc0-44e4-848c-4bf774d2fde6"). InnerVolumeSpecName "kube-api-access-hstrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:41:04.259513 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.259478    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d68738-4236-4ed2-8367-13bb27412e8f-kube-api-access-5lxkv" (OuterVolumeSpecName: "kube-api-access-5lxkv") pod "51d68738-4236-4ed2-8367-13bb27412e8f" (UID: "51d68738-4236-4ed2-8367-13bb27412e8f"). InnerVolumeSpecName "kube-api-access-5lxkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:41:04.359098 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.359058    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hstrw\" (UniqueName: \"kubernetes.io/projected/589e13de-5fc0-44e4-848c-4bf774d2fde6-kube-api-access-hstrw\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:04.359098 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.359093    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589e13de-5fc0-44e4-848c-4bf774d2fde6-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:04.359098 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.359103    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lxkv\" (UniqueName: \"kubernetes.io/projected/51d68738-4236-4ed2-8367-13bb27412e8f-kube-api-access-5lxkv\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:04.359098 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:04.359111    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51d68738-4236-4ed2-8367-13bb27412e8f-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:05.172231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.172200    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"
Sep 11 18:41:05.173888 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.173858    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9" event={"ID":"589e13de-5fc0-44e4-848c-4bf774d2fde6","Type":"ContainerDied","Data":"514cfa4c5ef52f7640ce0821a94dadbb332e8accd29dbf7308a95a74c3d41715"}
Sep 11 18:41:05.174039 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.173923    2578 scope.go:117] "RemoveContainer" containerID="36b1764aee17a62a0038043b09a60d80aa9417e66a15de49c7a3e87699c797c0"
Sep 11 18:41:05.174039 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.173870    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"
Sep 11 18:41:05.213925 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.213889    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"]
Sep 11 18:41:05.219390 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.219335    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-047453-c4f679d78-c78j9"]
Sep 11 18:41:05.229924 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.229894    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"]
Sep 11 18:41:05.236795 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.236764    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-047453-7cd478c9b9-74675"]
Sep 11 18:41:05.410321 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.410284    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="076ff3cd-dd7d-4abb-85cf-5bb996e41189" path="/var/lib/kubelet/pods/076ff3cd-dd7d-4abb-85cf-5bb996e41189/volumes"
Sep 11 18:41:05.410688 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.410664    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51d68738-4236-4ed2-8367-13bb27412e8f" path="/var/lib/kubelet/pods/51d68738-4236-4ed2-8367-13bb27412e8f/volumes"
Sep 11 18:41:05.411015 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:05.411003    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="589e13de-5fc0-44e4-848c-4bf774d2fde6" path="/var/lib/kubelet/pods/589e13de-5fc0-44e4-848c-4bf774d2fde6/volumes"
Sep 11 18:41:07.596927 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.596882    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk" podUID="aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" containerName="main" containerID="cri-o://18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee" gracePeriod=45
Sep 11 18:41:07.858978 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.858911    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:41:07.995305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995273    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tls-certs\") pod \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") "
Sep 11 18:41:07.995506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995325    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-model-cache\") pod \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") "
Sep 11 18:41:07.995506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995360    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-home\") pod \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") "
Sep 11 18:41:07.995506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995415    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-dshm\") pod \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") "
Sep 11 18:41:07.995506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995480    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5psc\" (UniqueName: \"kubernetes.io/projected/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-kube-api-access-l5psc\") pod \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") "
Sep 11 18:41:07.995506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995505    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tmp-dir\") pod \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\" (UID: \"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423\") "
Sep 11 18:41:07.995810 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995715    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-model-cache" (OuterVolumeSpecName: "model-cache") pod "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" (UID: "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:41:07.995810 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995730    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-home" (OuterVolumeSpecName: "home") pod "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" (UID: "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:41:07.995926 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995829    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" (UID: "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:41:07.995995 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995955    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tmp-dir\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:07.995995 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995971    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-model-cache\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:07.995995 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.995981    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-home\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:07.997746 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.997715    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-kube-api-access-l5psc" (OuterVolumeSpecName: "kube-api-access-l5psc") pod "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" (UID: "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423"). InnerVolumeSpecName "kube-api-access-l5psc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:41:07.998029 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.998010    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-dshm" (OuterVolumeSpecName: "dshm") pod "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" (UID: "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:41:07.998112 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:07.998019    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" (UID: "aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:41:08.096897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.096856    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-tls-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:08.096897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.096888    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-dshm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:08.096897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.096897    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5psc\" (UniqueName: \"kubernetes.io/projected/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423-kube-api-access-l5psc\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:08.191565 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.191534    2578 generic.go:356] "Generic (PLEG): container finished" podID="aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" containerID="18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee" exitCode=0
Sep 11 18:41:08.191741 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.191581    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"
Sep 11 18:41:08.191741 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.191616    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk" event={"ID":"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423","Type":"ContainerDied","Data":"18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee"}
Sep 11 18:41:08.191741 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.191647    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk" event={"ID":"aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423","Type":"ContainerDied","Data":"317174e06423ca6ea10d0e865943ca6c9ec1a0b0892a1a99d7764efa000471ae"}
Sep 11 18:41:08.191741 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.191663    2578 scope.go:117] "RemoveContainer" containerID="18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee"
Sep 11 18:41:08.201154 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.201132    2578 scope.go:117] "RemoveContainer" containerID="18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee"
Sep 11 18:41:08.201457 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:41:08.201439    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee\": container with ID starting with 18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee not found: ID does not exist" containerID="18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee"
Sep 11 18:41:08.201521 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.201466    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee"} err="failed to get container status \"18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee\": rpc error: code = NotFound desc = could not find container \"18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee\": container with ID starting with 18340a507b5645aad478fad9fd17a497e4d2c2021b25a26bc3e0ba149b24afee not found: ID does not exist"
Sep 11 18:41:08.216008 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.215970    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"]
Sep 11 18:41:08.219408 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:08.219378    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-047453/simulator-w0-047453-kserve-6df8db4596-l9mtk"]
Sep 11 18:41:09.410239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:09.410205    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" path="/var/lib/kubelet/pods/aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423/volumes"
Sep 11 18:41:11.731744 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:11.731671    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"]
Sep 11 18:41:11.732636 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:11.732597    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy" containerID="cri-o://47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be" gracePeriod=30
Sep 11 18:41:14.629702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.629240    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"]
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.629915    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d68738-4236-4ed2-8367-13bb27412e8f" containerName="payload-processing"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.629936    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d68738-4236-4ed2-8367-13bb27412e8f" containerName="payload-processing"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.629968    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="076ff3cd-dd7d-4abb-85cf-5bb996e41189" containerName="revoke-keys"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.629977    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="076ff3cd-dd7d-4abb-85cf-5bb996e41189" containerName="revoke-keys"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.629993    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589e13de-5fc0-44e4-848c-4bf774d2fde6" containerName="payload-pre-processing"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.630003    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="589e13de-5fc0-44e4-848c-4bf774d2fde6" containerName="payload-pre-processing"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.630020    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" containerName="main"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.630029    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" containerName="main"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.630136    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa39efd2-1cc7-4d1b-b67e-f7a5df7b2423" containerName="main"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.630148    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="51d68738-4236-4ed2-8367-13bb27412e8f" containerName="payload-processing"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.630159    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="589e13de-5fc0-44e4-848c-4bf774d2fde6" containerName="payload-pre-processing"
Sep 11 18:41:14.630231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.630168    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="076ff3cd-dd7d-4abb-85cf-5bb996e41189" containerName="revoke-keys"
Sep 11 18:41:14.644015 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.643981    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.649116 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.648180    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-553671-openshift-default-dockercfg-lbl6z\""
Sep 11 18:41:14.651063 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.650977    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"]
Sep 11 18:41:14.768727 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.768667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhgdw\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-kube-api-access-jhgdw\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.769084 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.769013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-credential-socket\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.769251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.769230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-token\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.769459 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.769443    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-data\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.769735 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.769714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istiod-ca-cert\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.769986 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.769963    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-socket\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.770170 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.770144    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-certs\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.770308 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.770290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-envoy\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.770497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.770480    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-podinfo\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.871712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.871653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-podinfo\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.872126 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.872094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhgdw\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-kube-api-access-jhgdw\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.872360 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.872336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-credential-socket\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.872582 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.872560    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-token\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.872793 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.872777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-data\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.873002 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.872984    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istiod-ca-cert\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.873137 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.873121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-socket\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.873252 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.873238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-certs\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.873361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.873347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-envoy\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.873794 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.873492    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-socket\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.873794 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.873148    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-data\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.873794 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.872845    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-credential-socket\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.873794 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.873789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istiod-ca-cert\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.874106 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.873837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-certs\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.875301 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.875258    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-podinfo\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.876200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.876182    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-envoy\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.897739 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.897658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-token\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.900383 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.900357    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhgdw\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-kube-api-access-jhgdw\") pod \"e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") " pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:14.922924 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.922854    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:41:14.923304 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.923258    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:41:14.958752 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:14.958715    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:15.260850 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:15.260817    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"]
Sep 11 18:41:15.262909 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:41:15.262877    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacddc5e8_23b0_4f9e_be54_28f3d2a09ca6.slice/crio-aab04e9adf9d83c4cd6121904a503e4d5e964764a48be34744cfc4f7beaa5d37 WatchSource:0}: Error finding container aab04e9adf9d83c4cd6121904a503e4d5e964764a48be34744cfc4f7beaa5d37: Status 404 returned error can't find the container with id aab04e9adf9d83c4cd6121904a503e4d5e964764a48be34744cfc4f7beaa5d37
Sep 11 18:41:15.265332 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:15.265301    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:41:15.265423 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:15.265369    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:41:15.265423 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:15.265397    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:41:16.252187 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:16.252152    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" event={"ID":"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6","Type":"ContainerStarted","Data":"ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf"}
Sep 11 18:41:16.252725 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:16.252214    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" event={"ID":"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6","Type":"ContainerStarted","Data":"aab04e9adf9d83c4cd6121904a503e4d5e964764a48be34744cfc4f7beaa5d37"}
Sep 11 18:41:16.276429 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:16.276307    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podStartSLOduration=2.2762877 podStartE2EDuration="2.2762877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:41:16.273780086 +0000 UTC m=+1539.308996758" watchObservedRunningTime="2026-09-11 18:41:16.2762877 +0000 UTC m=+1539.311504370"
Sep 11 18:41:16.959129 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:16.959086    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:16.964538 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:16.964508    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:17.010891 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.010813    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:41:17.199080 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199030    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-credential-socket\") pod \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") "
Sep 11 18:41:17.199282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199095    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgd6x\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-kube-api-access-bgd6x\") pod \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") "
Sep 11 18:41:17.199282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199140    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-socket\") pod \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") "
Sep 11 18:41:17.199282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199204    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-envoy\") pod \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") "
Sep 11 18:41:17.199282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199233    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-certs\") pod \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") "
Sep 11 18:41:17.199282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199258    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-token\") pod \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") "
Sep 11 18:41:17.199558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199294    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-data\") pod \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") "
Sep 11 18:41:17.199558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199346    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-podinfo\") pod \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") "
Sep 11 18:41:17.199558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199353    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" (UID: "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:41:17.199558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199400    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istiod-ca-cert\") pod \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\" (UID: \"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee\") "
Sep 11 18:41:17.199558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199506    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" (UID: "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:41:17.199832 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199710    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" (UID: "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:41:17.199832 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199710    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" (UID: "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:41:17.199978 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199952    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:17.200067 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199978    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:17.200067 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.199992    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:17.200067 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.200005    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:17.200067 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.200053    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" (UID: "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:41:17.202105 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.202069    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" (UID: "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:41:17.202216 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.202135    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" (UID: "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:41:17.202216 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.202158    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-kube-api-access-bgd6x" (OuterVolumeSpecName: "kube-api-access-bgd6x") pod "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" (UID: "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee"). InnerVolumeSpecName "kube-api-access-bgd6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:41:17.202313 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.202213    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" (UID: "e2f1cbe6-60fc-4c20-8b53-c4b9223427ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:41:17.261891 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.261800    2578 generic.go:356] "Generic (PLEG): container finished" podID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerID="47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be" exitCode=0
Sep 11 18:41:17.261891 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.261859    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"
Sep 11 18:41:17.262398 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.261890    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" event={"ID":"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee","Type":"ContainerDied","Data":"47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be"}
Sep 11 18:41:17.262398 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.261952    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8" event={"ID":"e2f1cbe6-60fc-4c20-8b53-c4b9223427ee","Type":"ContainerDied","Data":"959380fdb217017a55be6ba927a254858481db83d456f88e7fe738e2c60d817c"}
Sep 11 18:41:17.262398 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.261975    2578 scope.go:117] "RemoveContainer" containerID="47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be"
Sep 11 18:41:17.262398 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.262217    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:17.263829 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.263802    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:17.273372 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.273344    2578 scope.go:117] "RemoveContainer" containerID="83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7"
Sep 11 18:41:17.283005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.282983    2578 scope.go:117] "RemoveContainer" containerID="47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be"
Sep 11 18:41:17.283431 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:41:17.283410    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be\": container with ID starting with 47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be not found: ID does not exist" containerID="47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be"
Sep 11 18:41:17.283488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.283442    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be"} err="failed to get container status \"47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be\": rpc error: code = NotFound desc = could not find container \"47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be\": container with ID starting with 47462cc1e755306493cfb63802bc0b22d93f9223e168226fe5b873422ba7b4be not found: ID does not exist"
Sep 11 18:41:17.283488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.283465    2578 scope.go:117] "RemoveContainer" containerID="83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7"
Sep 11 18:41:17.283767 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:41:17.283752    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7\": container with ID starting with 83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7 not found: ID does not exist" containerID="83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7"
Sep 11 18:41:17.283831 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.283773    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7"} err="failed to get container status \"83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7\": rpc error: code = NotFound desc = could not find container \"83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7\": container with ID starting with 83053ff75ef3af5e28c7920cfbb8ab93e0dd5ae54e2a7a26ef9ca9620365e7c7 not found: ID does not exist"
Sep 11 18:41:17.300716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.300662    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgd6x\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-kube-api-access-bgd6x\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:17.300716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.300710    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:17.300716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.300724    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:17.300982 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.300733    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:17.300982 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.300747    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:17.319176 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.319106    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"]
Sep 11 18:41:17.332579 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.332524    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-047453-openshift-default-86bddf5b78-mmnn8"]
Sep 11 18:41:17.412862 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:17.412823    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" path="/var/lib/kubelet/pods/e2f1cbe6-60fc-4c20-8b53-c4b9223427ee/volumes"
Sep 11 18:41:19.083902 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:19.083861    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 18:41:19.084329 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:19.083937    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 18:41:20.283639 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:20.283608    2578 generic.go:356] "Generic (PLEG): container finished" podID="840dc218-df17-4626-b356-b01616169418" containerID="34f8df5ade8cc09f2f478ffe5906b1bebede3c1b902a063ab27cd6d41795d40e" exitCode=0
Sep 11 18:41:20.284099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:20.283646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" event={"ID":"840dc218-df17-4626-b356-b01616169418","Type":"ContainerDied","Data":"34f8df5ade8cc09f2f478ffe5906b1bebede3c1b902a063ab27cd6d41795d40e"}
Sep 11 18:41:20.284099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:20.284074    2578 scope.go:117] "RemoveContainer" containerID="34f8df5ade8cc09f2f478ffe5906b1bebede3c1b902a063ab27cd6d41795d40e"
Sep 11 18:41:20.285375 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:20.285343    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:41:20.285493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:20.285426    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:41:20.285493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:20.285477    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:41:21.291711 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:21.291659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" event={"ID":"840dc218-df17-4626-b356-b01616169418","Type":"ContainerStarted","Data":"60c2504a0262ca3fbacf7d348af1eed8a26bf0f19fc04ae21ac589b87e954b83"}
Sep 11 18:41:21.842444 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:21.842407    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:41:21.842800 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:21.842769    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 18:41:21.842876 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:21.842837    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 18:41:22.842653 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:22.842425    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 18:41:22.842653 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:22.842564    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 18:41:23.538075 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.538036    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"]
Sep 11 18:41:23.538538 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.538518    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy"
Sep 11 18:41:23.538638 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.538541    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy"
Sep 11 18:41:23.538638 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.538555    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy"
Sep 11 18:41:23.538638 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.538563    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy"
Sep 11 18:41:23.538815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.538660    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy"
Sep 11 18:41:23.538815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.538694    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2f1cbe6-60fc-4c20-8b53-c4b9223427ee" containerName="istio-proxy"
Sep 11 18:41:23.541517 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.541497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:23.544255 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.544235    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-553671-dockercfg-kwmbm\""
Sep 11 18:41:23.544374 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.544237    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-553671\""
Sep 11 18:41:23.554836 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.554808    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"]
Sep 11 18:41:23.559176 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.559150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/451892e8-e092-45c3-b2d5-a3353a84efe7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8\" (UID: \"451892e8-e092-45c3-b2d5-a3353a84efe7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:23.559304 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.559239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr8l6\" (UniqueName: \"kubernetes.io/projected/451892e8-e092-45c3-b2d5-a3353a84efe7-kube-api-access-hr8l6\") pod \"payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8\" (UID: \"451892e8-e092-45c3-b2d5-a3353a84efe7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:23.661277 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.661230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr8l6\" (UniqueName: \"kubernetes.io/projected/451892e8-e092-45c3-b2d5-a3353a84efe7-kube-api-access-hr8l6\") pod \"payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8\" (UID: \"451892e8-e092-45c3-b2d5-a3353a84efe7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:23.661637 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.661619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/451892e8-e092-45c3-b2d5-a3353a84efe7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8\" (UID: \"451892e8-e092-45c3-b2d5-a3353a84efe7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:23.662767 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.662560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/451892e8-e092-45c3-b2d5-a3353a84efe7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8\" (UID: \"451892e8-e092-45c3-b2d5-a3353a84efe7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:23.671426 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.671394    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr8l6\" (UniqueName: \"kubernetes.io/projected/451892e8-e092-45c3-b2d5-a3353a84efe7-kube-api-access-hr8l6\") pod \"payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8\" (UID: \"451892e8-e092-45c3-b2d5-a3353a84efe7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:23.842457 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.842366    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 18:41:23.842457 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.842427    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 18:41:23.851583 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:23.851556    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:24.144828 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:24.144748    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"]
Sep 11 18:41:24.148319 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:41:24.148288    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod451892e8_e092_45c3_b2d5_a3353a84efe7.slice/crio-075ab674226d958727a5fa7543a94d64cff26efa6a5d2392e8a77c7db7dda2f0 WatchSource:0}: Error finding container 075ab674226d958727a5fa7543a94d64cff26efa6a5d2392e8a77c7db7dda2f0: Status 404 returned error can't find the container with id 075ab674226d958727a5fa7543a94d64cff26efa6a5d2392e8a77c7db7dda2f0
Sep 11 18:41:24.310786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:24.310741    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8" event={"ID":"451892e8-e092-45c3-b2d5-a3353a84efe7","Type":"ContainerStarted","Data":"5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9"}
Sep 11 18:41:24.310999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:24.310815    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8" event={"ID":"451892e8-e092-45c3-b2d5-a3353a84efe7","Type":"ContainerStarted","Data":"075ab674226d958727a5fa7543a94d64cff26efa6a5d2392e8a77c7db7dda2f0"}
Sep 11 18:41:24.311669 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:24.311639    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:24.842430 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:24.842386    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 18:41:24.842633 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:24.842446    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 18:41:25.842577 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:25.842540    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 11 18:41:25.842987 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:25.842599    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 11 18:41:26.849644 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:26.849605    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:41:26.850180 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:26.850156    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:41:26.850870 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:26.850849    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:41:26.875082 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:26.875032    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8" podStartSLOduration=3.875014988 podStartE2EDuration="3.875014988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:41:24.333030828 +0000 UTC m=+1547.368247499" watchObservedRunningTime="2026-09-11 18:41:26.875014988 +0000 UTC m=+1549.910231658"
Sep 11 18:41:36.330705 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:36.330644    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:41:37.649958 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:37.649918    2578 scope.go:117] "RemoveContainer" containerID="6ae48d456edac0a10b13bbfc9bc1b59e270a98c02124ba3aaaae6e9a263bf9a4"
Sep 11 18:41:47.264052 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:47.264014    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 11 18:41:47.264499 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:47.264101    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 11 18:41:47.475299 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:47.475266    2578 generic.go:356] "Generic (PLEG): container finished" podID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerID="ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf" exitCode=0
Sep 11 18:41:47.475492 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:47.475350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" event={"ID":"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6","Type":"ContainerDied","Data":"ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf"}
Sep 11 18:41:47.475832 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:47.475815    2578 scope.go:117] "RemoveContainer" containerID="ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf"
Sep 11 18:41:47.476963 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:47.476935    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:41:47.477061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:47.476998    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:41:47.477061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:47.477028    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:41:48.483800 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:48.483762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" event={"ID":"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6","Type":"ContainerStarted","Data":"0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06"}
Sep 11 18:41:48.959309 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:48.959267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:48.959527 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:48.959497    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 11 18:41:48.959594 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:48.959570    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 11 18:41:49.959852 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:49.959819    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 11 18:41:49.960245 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:49.959884    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 11 18:41:50.616431 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.616380    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"]
Sep 11 18:41:50.620417 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.620382    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"
Sep 11 18:41:50.625014 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.624986    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:41:50.625283 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.625004    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:41:50.632996 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.632343    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"]
Sep 11 18:41:50.744528 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.744488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9c9eab-3feb-4a85-9d38-66f9a5cac81f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-699caf96-cpwgh\" (UID: \"6f9c9eab-3feb-4a85-9d38-66f9a5cac81f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"
Sep 11 18:41:50.845954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.845920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9c9eab-3feb-4a85-9d38-66f9a5cac81f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-699caf96-cpwgh\" (UID: \"6f9c9eab-3feb-4a85-9d38-66f9a5cac81f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"
Sep 11 18:41:50.846787 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.846753    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9c9eab-3feb-4a85-9d38-66f9a5cac81f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-699caf96-cpwgh\" (UID: \"6f9c9eab-3feb-4a85-9d38-66f9a5cac81f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"
Sep 11 18:41:50.938666 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.938618    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"
Sep 11 18:41:50.959497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.959463    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 11 18:41:50.959664 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:50.959529    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 11 18:41:51.233407 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:51.229345    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"]
Sep 11 18:41:51.233914 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:41:51.233864    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9c9eab_3feb_4a85_9d38_66f9a5cac81f.slice/crio-32e889249092d750fc52167bf563a4d0294b1465f8cf6c48fa034b4d34007910 WatchSource:0}: Error finding container 32e889249092d750fc52167bf563a4d0294b1465f8cf6c48fa034b4d34007910: Status 404 returned error can't find the container with id 32e889249092d750fc52167bf563a4d0294b1465f8cf6c48fa034b4d34007910
Sep 11 18:41:51.502099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:51.502017    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh" event={"ID":"6f9c9eab-3feb-4a85-9d38-66f9a5cac81f","Type":"ContainerStarted","Data":"32e889249092d750fc52167bf563a4d0294b1465f8cf6c48fa034b4d34007910"}
Sep 11 18:41:51.959166 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:51.959130    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 11 18:41:51.959339 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:51.959195    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 11 18:41:52.509783 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:52.509286    2578 generic.go:356] "Generic (PLEG): container finished" podID="6f9c9eab-3feb-4a85-9d38-66f9a5cac81f" containerID="72d910f60a65240152b5522b1c4bc30bd325a18bd9509fe63f6067511cc50b9a" exitCode=0
Sep 11 18:41:52.510415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:52.509358    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh" event={"ID":"6f9c9eab-3feb-4a85-9d38-66f9a5cac81f","Type":"ContainerDied","Data":"72d910f60a65240152b5522b1c4bc30bd325a18bd9509fe63f6067511cc50b9a"}
Sep 11 18:41:52.959401 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:52.959356    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 11 18:41:52.959626 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:52.959425    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 11 18:41:53.704440 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:53.704410    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"
Sep 11 18:41:53.779412 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:53.779330    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9c9eab-3feb-4a85-9d38-66f9a5cac81f-maas-api-service-ca\") pod \"6f9c9eab-3feb-4a85-9d38-66f9a5cac81f\" (UID: \"6f9c9eab-3feb-4a85-9d38-66f9a5cac81f\") "
Sep 11 18:41:53.779721 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:53.779697    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f9c9eab-3feb-4a85-9d38-66f9a5cac81f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f9c9eab-3feb-4a85-9d38-66f9a5cac81f" (UID: "6f9c9eab-3feb-4a85-9d38-66f9a5cac81f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:41:53.880568 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:53.880528    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f9c9eab-3feb-4a85-9d38-66f9a5cac81f-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:41:53.959295 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:53.959259    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 11 18:41:53.959467 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:53.959324    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 11 18:41:54.530065 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:54.530013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh" event={"ID":"6f9c9eab-3feb-4a85-9d38-66f9a5cac81f","Type":"ContainerDied","Data":"32e889249092d750fc52167bf563a4d0294b1465f8cf6c48fa034b4d34007910"}
Sep 11 18:41:54.530280 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:54.530087    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32e889249092d750fc52167bf563a4d0294b1465f8cf6c48fa034b4d34007910"
Sep 11 18:41:54.530280 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:54.530157    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"
Sep 11 18:41:54.959367 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:54.959319    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 11 18:41:54.959796 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:54.959408    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 11 18:41:55.963790 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:55.963756    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:55.964547 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:55.964175    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:41:55.964865 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:41:55.964847    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:42:06.392972 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.392932    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"]
Sep 11 18:42:06.393459 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.393205    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm" podUID="747a6523-2d9e-40ed-93f2-faa422788dcc" containerName="payload-processing" containerID="cri-o://5f2265d5a19225791c9890b1b20fe54ebe216e1fc044c9155c57ba126f0f7232" gracePeriod=30
Sep 11 18:42:06.596979 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.596939    2578 generic.go:356] "Generic (PLEG): container finished" podID="747a6523-2d9e-40ed-93f2-faa422788dcc" containerID="5f2265d5a19225791c9890b1b20fe54ebe216e1fc044c9155c57ba126f0f7232" exitCode=0
Sep 11 18:42:06.597235 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.597200    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm" event={"ID":"747a6523-2d9e-40ed-93f2-faa422788dcc","Type":"ContainerDied","Data":"5f2265d5a19225791c9890b1b20fe54ebe216e1fc044c9155c57ba126f0f7232"}
Sep 11 18:42:06.601031 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.600983    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"]
Sep 11 18:42:06.606174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.606148    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-699caf96-cpwgh"]
Sep 11 18:42:06.657745 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.657667    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:42:06.707438 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.707402    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtjcq\" (UniqueName: \"kubernetes.io/projected/747a6523-2d9e-40ed-93f2-faa422788dcc-kube-api-access-vtjcq\") pod \"747a6523-2d9e-40ed-93f2-faa422788dcc\" (UID: \"747a6523-2d9e-40ed-93f2-faa422788dcc\") "
Sep 11 18:42:06.707578 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.707516    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/747a6523-2d9e-40ed-93f2-faa422788dcc-plugins-config-volume\") pod \"747a6523-2d9e-40ed-93f2-faa422788dcc\" (UID: \"747a6523-2d9e-40ed-93f2-faa422788dcc\") "
Sep 11 18:42:06.707879 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.707857    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/747a6523-2d9e-40ed-93f2-faa422788dcc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "747a6523-2d9e-40ed-93f2-faa422788dcc" (UID: "747a6523-2d9e-40ed-93f2-faa422788dcc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:42:06.709567 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.709537    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/747a6523-2d9e-40ed-93f2-faa422788dcc-kube-api-access-vtjcq" (OuterVolumeSpecName: "kube-api-access-vtjcq") pod "747a6523-2d9e-40ed-93f2-faa422788dcc" (UID: "747a6523-2d9e-40ed-93f2-faa422788dcc"). InnerVolumeSpecName "kube-api-access-vtjcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:42:06.808748 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.808671    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtjcq\" (UniqueName: \"kubernetes.io/projected/747a6523-2d9e-40ed-93f2-faa422788dcc-kube-api-access-vtjcq\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:42:06.808748 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:06.808737    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/747a6523-2d9e-40ed-93f2-faa422788dcc-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:42:07.410437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:07.410404    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f9c9eab-3feb-4a85-9d38-66f9a5cac81f" path="/var/lib/kubelet/pods/6f9c9eab-3feb-4a85-9d38-66f9a5cac81f/volumes"
Sep 11 18:42:07.604227 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:07.604193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm" event={"ID":"747a6523-2d9e-40ed-93f2-faa422788dcc","Type":"ContainerDied","Data":"242c4e84aab6f4fab02102d318984f6082e30f86c92c85a9e171b02df79951aa"}
Sep 11 18:42:07.604227 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:07.604220    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"
Sep 11 18:42:07.604454 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:07.604251    2578 scope.go:117] "RemoveContainer" containerID="5f2265d5a19225791c9890b1b20fe54ebe216e1fc044c9155c57ba126f0f7232"
Sep 11 18:42:07.622569 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:07.622538    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"]
Sep 11 18:42:07.627484 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:07.627462    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-699caf96-865b55fc45-2fzlm"]
Sep 11 18:42:09.410072 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:09.410039    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="747a6523-2d9e-40ed-93f2-faa422788dcc" path="/var/lib/kubelet/pods/747a6523-2d9e-40ed-93f2-faa422788dcc/volumes"
Sep 11 18:42:14.723936 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:14.723901    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"]
Sep 11 18:42:16.344552 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.344470    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"]
Sep 11 18:42:16.345007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.344864    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9c9eab-3feb-4a85-9d38-66f9a5cac81f" containerName="revoke-keys"
Sep 11 18:42:16.345007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.344876    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9c9eab-3feb-4a85-9d38-66f9a5cac81f" containerName="revoke-keys"
Sep 11 18:42:16.345007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.344898    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747a6523-2d9e-40ed-93f2-faa422788dcc" containerName="payload-processing"
Sep 11 18:42:16.345007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.344908    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="747a6523-2d9e-40ed-93f2-faa422788dcc" containerName="payload-processing"
Sep 11 18:42:16.345007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.344985    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9c9eab-3feb-4a85-9d38-66f9a5cac81f" containerName="revoke-keys"
Sep 11 18:42:16.345007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.345002    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="747a6523-2d9e-40ed-93f2-faa422788dcc" containerName="payload-processing"
Sep 11 18:42:16.348223 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.348206    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.351515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.351487    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-d90e71\"/\"openshift-service-ca.crt\""
Sep 11 18:42:16.351515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.351510    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-d90e71\"/\"default-dockercfg-p9zhk\""
Sep 11 18:42:16.351708 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.351487    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-d90e71\"/\"auth-test-model-d90e71-kserve-self-signed-certs\""
Sep 11 18:42:16.351708 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.351578    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-d90e71\"/\"kube-root-ca.crt\""
Sep 11 18:42:16.359251 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.359229    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"]
Sep 11 18:42:16.398355 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.398324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-dshm\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.398545 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.398358    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-tmp-dir\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.398545 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.398405    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d03be4dc-8b14-42d9-af84-673967b7af14-tls-certs\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.398545 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.398462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5s6g\" (UniqueName: \"kubernetes.io/projected/d03be4dc-8b14-42d9-af84-673967b7af14-kube-api-access-h5s6g\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.398730 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.398578    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-model-cache\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.398730 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.398617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-home\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.500028 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.499993    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-model-cache\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.500208 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.500152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-home\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.500822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.500335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-dshm\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.500822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.500377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-tmp-dir\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.500822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.500477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d03be4dc-8b14-42d9-af84-673967b7af14-tls-certs\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.500822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.500509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5s6g\" (UniqueName: \"kubernetes.io/projected/d03be4dc-8b14-42d9-af84-673967b7af14-kube-api-access-h5s6g\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.500822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.500546    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-home\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.500822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.500591    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-model-cache\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.500822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.500775    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-tmp-dir\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.502599 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.502573    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-dshm\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.503143 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.503121    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d03be4dc-8b14-42d9-af84-673967b7af14-tls-certs\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.509239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.509217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5s6g\" (UniqueName: \"kubernetes.io/projected/d03be4dc-8b14-42d9-af84-673967b7af14-kube-api-access-h5s6g\") pod \"auth-test-model-d90e71-kserve-84cb978c-nh7wq\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") " pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.657716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.657614    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:16.933974 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:16.933948    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"]
Sep 11 18:42:16.936006 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:42:16.935971    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd03be4dc_8b14_42d9_af84_673967b7af14.slice/crio-a9f70679519254029d16cb436efa2716faaeef44b83033dffb82f1b43f6d4c47 WatchSource:0}: Error finding container a9f70679519254029d16cb436efa2716faaeef44b83033dffb82f1b43f6d4c47: Status 404 returned error can't find the container with id a9f70679519254029d16cb436efa2716faaeef44b83033dffb82f1b43f6d4c47
Sep 11 18:42:17.657593 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:17.657549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq" event={"ID":"d03be4dc-8b14-42d9-af84-673967b7af14","Type":"ContainerStarted","Data":"7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9"}
Sep 11 18:42:17.658041 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:17.657603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq" event={"ID":"d03be4dc-8b14-42d9-af84-673967b7af14","Type":"ContainerStarted","Data":"a9f70679519254029d16cb436efa2716faaeef44b83033dffb82f1b43f6d4c47"}
Sep 11 18:42:17.674737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:17.674650    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq" podStartSLOduration=1.6746315040000002 podStartE2EDuration="1.674631504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:42:17.674048217 +0000 UTC m=+1600.709264888" watchObservedRunningTime="2026-09-11 18:42:17.674631504 +0000 UTC m=+1600.709848178"
Sep 11 18:42:26.658616 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:26.658570    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:26.658616 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:26.658618    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:26.670960 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:26.670936    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:26.719256 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:26.719224    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:42:29.743161 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:29.743115    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx" podUID="5208bf9d-1889-4892-81c7-5ccc25137746" containerName="main" containerID="cri-o://90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d" gracePeriod=45
Sep 11 18:42:29.992159 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:29.992134    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:42:30.129259 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129175    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-home\") pod \"5208bf9d-1889-4892-81c7-5ccc25137746\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") "
Sep 11 18:42:30.129259 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129236    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-dshm\") pod \"5208bf9d-1889-4892-81c7-5ccc25137746\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") "
Sep 11 18:42:30.129490 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129260    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm9pm\" (UniqueName: \"kubernetes.io/projected/5208bf9d-1889-4892-81c7-5ccc25137746-kube-api-access-qm9pm\") pod \"5208bf9d-1889-4892-81c7-5ccc25137746\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") "
Sep 11 18:42:30.129490 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129278    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-tmp-dir\") pod \"5208bf9d-1889-4892-81c7-5ccc25137746\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") "
Sep 11 18:42:30.129490 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129308    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-model-cache\") pod \"5208bf9d-1889-4892-81c7-5ccc25137746\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") "
Sep 11 18:42:30.129490 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129324    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5208bf9d-1889-4892-81c7-5ccc25137746-tls-certs\") pod \"5208bf9d-1889-4892-81c7-5ccc25137746\" (UID: \"5208bf9d-1889-4892-81c7-5ccc25137746\") "
Sep 11 18:42:30.129490 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129457    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-home" (OuterVolumeSpecName: "home") pod "5208bf9d-1889-4892-81c7-5ccc25137746" (UID: "5208bf9d-1889-4892-81c7-5ccc25137746"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:42:30.129851 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129594    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5208bf9d-1889-4892-81c7-5ccc25137746" (UID: "5208bf9d-1889-4892-81c7-5ccc25137746"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:42:30.129851 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129618    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-model-cache" (OuterVolumeSpecName: "model-cache") pod "5208bf9d-1889-4892-81c7-5ccc25137746" (UID: "5208bf9d-1889-4892-81c7-5ccc25137746"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:42:30.129851 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129775    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-home\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:42:30.129851 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129797    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-tmp-dir\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:42:30.129851 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.129811    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-model-cache\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:42:30.131594 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.131570    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-dshm" (OuterVolumeSpecName: "dshm") pod "5208bf9d-1889-4892-81c7-5ccc25137746" (UID: "5208bf9d-1889-4892-81c7-5ccc25137746"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:42:30.131724 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.131590    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5208bf9d-1889-4892-81c7-5ccc25137746-kube-api-access-qm9pm" (OuterVolumeSpecName: "kube-api-access-qm9pm") pod "5208bf9d-1889-4892-81c7-5ccc25137746" (UID: "5208bf9d-1889-4892-81c7-5ccc25137746"). InnerVolumeSpecName "kube-api-access-qm9pm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:42:30.131724 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.131605    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5208bf9d-1889-4892-81c7-5ccc25137746-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5208bf9d-1889-4892-81c7-5ccc25137746" (UID: "5208bf9d-1889-4892-81c7-5ccc25137746"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:42:30.231138 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.231101    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5208bf9d-1889-4892-81c7-5ccc25137746-dshm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:42:30.231138 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.231132    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm9pm\" (UniqueName: \"kubernetes.io/projected/5208bf9d-1889-4892-81c7-5ccc25137746-kube-api-access-qm9pm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:42:30.231138 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.231143    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5208bf9d-1889-4892-81c7-5ccc25137746-tls-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:42:30.731415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.731383    2578 generic.go:356] "Generic (PLEG): container finished" podID="5208bf9d-1889-4892-81c7-5ccc25137746" containerID="90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d" exitCode=0
Sep 11 18:42:30.731587 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.731421    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"
Sep 11 18:42:30.731587 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.731466    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx" event={"ID":"5208bf9d-1889-4892-81c7-5ccc25137746","Type":"ContainerDied","Data":"90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d"}
Sep 11 18:42:30.731587 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.731508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx" event={"ID":"5208bf9d-1889-4892-81c7-5ccc25137746","Type":"ContainerDied","Data":"0972c45b84ceefa82eb86d51c774aed5f2d2dc3b662f5a1e8abc28831b34f15f"}
Sep 11 18:42:30.731587 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.731526    2578 scope.go:117] "RemoveContainer" containerID="90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d"
Sep 11 18:42:30.741234 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.741215    2578 scope.go:117] "RemoveContainer" containerID="90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d"
Sep 11 18:42:30.741491 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:42:30.741471    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d\": container with ID starting with 90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d not found: ID does not exist" containerID="90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d"
Sep 11 18:42:30.741542 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.741499    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d"} err="failed to get container status \"90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d\": rpc error: code = NotFound desc = could not find container \"90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d\": container with ID starting with 90794b49e75a1e2098ba8f3d1d303d81f396b9213b5a2f7b88c1285683f59d7d not found: ID does not exist"
Sep 11 18:42:30.752822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.752797    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"]
Sep 11 18:42:30.758713 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:30.758693    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-699caf96/e2e-lifecycle-model-699caf96-kserve-69b5995555-9jbmx"]
Sep 11 18:42:31.411408 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:42:31.411374    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5208bf9d-1889-4892-81c7-5ccc25137746" path="/var/lib/kubelet/pods/5208bf9d-1889-4892-81c7-5ccc25137746/volumes"
Sep 11 18:43:37.705725 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:37.705692    2578 scope.go:117] "RemoveContainer" containerID="d31cffa7e44e9e78b0e978e57f02cf1dfe3a9818acb5f610e4cc3012bf2103c7"
Sep 11 18:43:37.714186 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:37.714163    2578 scope.go:117] "RemoveContainer" containerID="76dfa66ee348fb0fa1e463e138804cd7752345ce0935f92591c69357d654d358"
Sep 11 18:43:39.055438 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.055405    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"]
Sep 11 18:43:39.055832 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.055801    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5208bf9d-1889-4892-81c7-5ccc25137746" containerName="main"
Sep 11 18:43:39.055832 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.055814    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5208bf9d-1889-4892-81c7-5ccc25137746" containerName="main"
Sep 11 18:43:39.055903 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.055894    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5208bf9d-1889-4892-81c7-5ccc25137746" containerName="main"
Sep 11 18:43:39.060334 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.060318    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.064054 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.064032    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"default-dockercfg-mfbjv\""
Sep 11 18:43:39.064195 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.064119    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"kube-root-ca.crt\""
Sep 11 18:43:39.064195 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.064120    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"openshift-service-ca.crt\""
Sep 11 18:43:39.064323 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.064211    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"auth-test-model-553671-kserve-self-signed-certs\""
Sep 11 18:43:39.071787 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.071765    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"]
Sep 11 18:43:39.189955 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.189917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-dshm\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.190115 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.189975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-model-cache\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.190115 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.190056    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tls-certs\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.190115 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.190091    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr2zj\" (UniqueName: \"kubernetes.io/projected/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-kube-api-access-wr2zj\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.190219 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.190174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tmp-dir\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.190253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.190234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-home\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.291561 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.291528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-dshm\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.291738 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.291576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-model-cache\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.291738 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.291610    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tls-certs\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.291738 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.291635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr2zj\" (UniqueName: \"kubernetes.io/projected/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-kube-api-access-wr2zj\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.291738 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.291671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tmp-dir\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.291892 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.291750    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-home\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.292087 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.292054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-model-cache\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.292228 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.292096    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-home\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.292228 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.292114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tmp-dir\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.293813 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.293783    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-dshm\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.294061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.294045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tls-certs\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.300489 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.300469    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr2zj\" (UniqueName: \"kubernetes.io/projected/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-kube-api-access-wr2zj\") pod \"auth-test-model-553671-kserve-6c5b8b9f79-sspn6\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") " pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.369035 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.368942    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:39.641285 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:39.641211    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"]
Sep 11 18:43:39.644209 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:43:39.644182    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f940308_7457_4ef9_8d5e_2b4c7cdbfbb0.slice/crio-3c2174498ae5f539abf5042b4adc5312fe4b2cc88197190fbba1f5e1033bc9c5 WatchSource:0}: Error finding container 3c2174498ae5f539abf5042b4adc5312fe4b2cc88197190fbba1f5e1033bc9c5: Status 404 returned error can't find the container with id 3c2174498ae5f539abf5042b4adc5312fe4b2cc88197190fbba1f5e1033bc9c5
Sep 11 18:43:40.119229 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:40.119189    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6" event={"ID":"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0","Type":"ContainerStarted","Data":"80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9"}
Sep 11 18:43:40.119229 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:40.119236    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6" event={"ID":"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0","Type":"ContainerStarted","Data":"3c2174498ae5f539abf5042b4adc5312fe4b2cc88197190fbba1f5e1033bc9c5"}
Sep 11 18:43:40.137295 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:40.137237    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6" podStartSLOduration=1.1372201020000001 podStartE2EDuration="1.137220102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:43:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:43:40.134917701 +0000 UTC m=+1683.170134369" watchObservedRunningTime="2026-09-11 18:43:40.137220102 +0000 UTC m=+1683.172436771"
Sep 11 18:43:49.369078 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:49.369038    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:49.369078 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:49.369090    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:49.381426 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:49.381402    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:43:50.181700 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:43:50.181662    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:44:07.859032 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.858929    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"]
Sep 11 18:44:07.863490 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.863467    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:07.867573 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.867548    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-883e85-openshift-default-dockercfg-ggnck\""
Sep 11 18:44:07.877038 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.877010    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"]
Sep 11 18:44:07.966585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.966548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-token\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:07.966897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.966871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-credential-socket\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:07.967090 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.967070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istiod-ca-cert\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:07.967302 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.967284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-certs\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:07.967453 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.967438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-podinfo\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:07.967617 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.967599    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc4j6\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-kube-api-access-wc4j6\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:07.967824 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.967808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-envoy\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:07.968013 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.967996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-socket\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:07.968173 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:07.968149    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-data\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.069596 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.069561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-token\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.069799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.069622    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-credential-socket\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.069799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.069644    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istiod-ca-cert\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.069799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.069713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-certs\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.069799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.069730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-podinfo\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.069799 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.069748    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc4j6\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-kube-api-access-wc4j6\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.070102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.069814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-envoy\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.070102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.069862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-socket\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.070102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.069897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-data\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.070244 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.070119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-credential-socket\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.070355 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.070331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-socket\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.070420 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.070382    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-data\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.070420 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.070380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-certs\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.070590 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.070572    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istiod-ca-cert\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.072021 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.072000    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-envoy\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.072327 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.072304    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-podinfo\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.077996 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.077973    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-token\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.078274 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.078252    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc4j6\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-kube-api-access-wc4j6\") pod \"e2e-api-a-883e85-openshift-default-8446658c86-mkmlq\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") " pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.178815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.178785    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:08.455735 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.455706    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"]
Sep 11 18:44:08.457509 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:44:08.457480    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb6fd2b2_a2a3_479b_a9f6_ccf45a96f95a.slice/crio-025665dea957967bcf9b7645c7056c98f0dae2bea4541300114e969b2171d253 WatchSource:0}: Error finding container 025665dea957967bcf9b7645c7056c98f0dae2bea4541300114e969b2171d253: Status 404 returned error can't find the container with id 025665dea957967bcf9b7645c7056c98f0dae2bea4541300114e969b2171d253
Sep 11 18:44:08.459819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.459788    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:44:08.459907 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.459850    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:44:08.459907 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:08.459877    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:44:09.277931 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:09.277890    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" event={"ID":"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a","Type":"ContainerStarted","Data":"921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700"}
Sep 11 18:44:09.278417 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:09.277982    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" event={"ID":"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a","Type":"ContainerStarted","Data":"025665dea957967bcf9b7645c7056c98f0dae2bea4541300114e969b2171d253"}
Sep 11 18:44:09.313424 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:09.313337    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" podStartSLOduration=2.313318625 podStartE2EDuration="2.313318625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:44:09.312265553 +0000 UTC m=+1712.347482222" watchObservedRunningTime="2026-09-11 18:44:09.313318625 +0000 UTC m=+1712.348535295"
Sep 11 18:44:10.179866 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:10.179832    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:10.184755 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:10.184730    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:10.282910 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:10.282873    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:10.283874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:10.283850    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:35.431531 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:35.431494    2578 generic.go:356] "Generic (PLEG): container finished" podID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerID="921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700" exitCode=0
Sep 11 18:44:35.431990 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:35.431558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" event={"ID":"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a","Type":"ContainerDied","Data":"921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700"}
Sep 11 18:44:35.432057 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:35.432042    2578 scope.go:117] "RemoveContainer" containerID="921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700"
Sep 11 18:44:35.433192 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:35.433158    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:44:35.433305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:35.433237    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:44:35.433305 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:35.433283    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:44:36.443046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:36.443007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" event={"ID":"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a","Type":"ContainerStarted","Data":"d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76"}
Sep 11 18:44:37.179389 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:37.179346    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:37.180542 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:37.180515    2578 patch_prober.go:28] interesting pod/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 11 18:44:37.180669 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:37.180566    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 11 18:44:38.179185 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:38.179143    2578 patch_prober.go:28] interesting pod/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 11 18:44:38.179582 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:38.179213    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 11 18:44:39.180103 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:39.180063    2578 patch_prober.go:28] interesting pod/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 11 18:44:39.180501 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:39.180127    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 11 18:44:40.179708 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:40.179639    2578 patch_prober.go:28] interesting pod/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 11 18:44:40.179920 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:40.179722    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 11 18:44:40.283560 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:40.283525    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:41.179809 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:41.179769    2578 patch_prober.go:28] interesting pod/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 11 18:44:41.180007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:41.179826    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 11 18:44:41.982372 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:41.982335    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"]
Sep 11 18:44:41.986309 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:41.986278    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:41.990253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:41.990225    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-5cbc66-openshift-default-dockercfg-vmrst\""
Sep 11 18:44:41.998552 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:41.998521    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"]
Sep 11 18:44:42.116748 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.116704    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8030c458-6d3d-44ed-97ca-a92701958f74-istiod-ca-cert\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.117019 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.117001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-istio-token\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.117181 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.117166    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-envoy\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.117366 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.117352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-credential-socket\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.117515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.117502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-certs\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.117662 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.117649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8030c458-6d3d-44ed-97ca-a92701958f74-istio-podinfo\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.117853 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.117836    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpjzg\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-kube-api-access-cpjzg\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.117983 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.117969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-data\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.118118 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.118103    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-socket\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.187447 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.187409    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:42.188672 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.188647    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:44:42.219394 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-credential-socket\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219570 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219414    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-certs\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219570 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8030c458-6d3d-44ed-97ca-a92701958f74-istio-podinfo\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219570 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpjzg\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-kube-api-access-cpjzg\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219570 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-data\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219768 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-socket\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219863 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219841    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8030c458-6d3d-44ed-97ca-a92701958f74-istiod-ca-cert\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219947 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219863    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-credential-socket\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219947 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-istio-token\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219947 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-data\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.219947 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-envoy\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.220145 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.219963    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-certs\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.220145 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.220001    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-socket\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.220526 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.220502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8030c458-6d3d-44ed-97ca-a92701958f74-istiod-ca-cert\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.221964 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.221943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-envoy\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.222127 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.222109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8030c458-6d3d-44ed-97ca-a92701958f74-istio-podinfo\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.232595 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.232513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-istio-token\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.232770 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.232754    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpjzg\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-kube-api-access-cpjzg\") pod \"e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") " pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.300078 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.300038    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:42.583086 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.583060    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"]
Sep 11 18:44:42.584814 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:44:42.584779    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8030c458_6d3d_44ed_97ca_a92701958f74.slice/crio-e1f9ebad9e5c7caf89754ef6e37fbdfc8713ccdef1613b0e9f77e2eea1ae7e16 WatchSource:0}: Error finding container e1f9ebad9e5c7caf89754ef6e37fbdfc8713ccdef1613b0e9f77e2eea1ae7e16: Status 404 returned error can't find the container with id e1f9ebad9e5c7caf89754ef6e37fbdfc8713ccdef1613b0e9f77e2eea1ae7e16
Sep 11 18:44:42.587023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.586992    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:44:42.587104 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.587083    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:44:42.587146 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:42.587115    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:44:43.487027 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:43.486990    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" event={"ID":"8030c458-6d3d-44ed-97ca-a92701958f74","Type":"ContainerStarted","Data":"f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4"}
Sep 11 18:44:43.487420 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:43.487044    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" event={"ID":"8030c458-6d3d-44ed-97ca-a92701958f74","Type":"ContainerStarted","Data":"e1f9ebad9e5c7caf89754ef6e37fbdfc8713ccdef1613b0e9f77e2eea1ae7e16"}
Sep 11 18:44:43.510817 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:43.510767    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" podStartSLOduration=2.510750567 podStartE2EDuration="2.510750567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:44:43.507845228 +0000 UTC m=+1746.543061907" watchObservedRunningTime="2026-09-11 18:44:43.510750567 +0000 UTC m=+1746.545967236"
Sep 11 18:44:44.300597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:44.300561    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:44.305815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:44.305792    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:44.492584 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:44.492554    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:44.493480 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:44.493460    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:44:51.109722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.109685    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"]
Sep 11 18:44:51.112339 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.112322    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:44:51.115162 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.115139    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-5cbc66-dockercfg-jn6mk\""
Sep 11 18:44:51.115300 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.115170    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-5cbc66\""
Sep 11 18:44:51.122745 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.122719    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"]
Sep 11 18:44:51.210933 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.210882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvgpl\" (UniqueName: \"kubernetes.io/projected/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-kube-api-access-pvgpl\") pod \"payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz\" (UID: \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:44:51.211139 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.211007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz\" (UID: \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:44:51.312194 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.312150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz\" (UID: \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:44:51.312376 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.312305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvgpl\" (UniqueName: \"kubernetes.io/projected/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-kube-api-access-pvgpl\") pod \"payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz\" (UID: \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:44:51.312944 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.312926    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz\" (UID: \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:44:51.320475 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.320454    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvgpl\" (UniqueName: \"kubernetes.io/projected/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-kube-api-access-pvgpl\") pod \"payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz\" (UID: \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:44:51.423437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.423400    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:44:51.699879 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:51.699856    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"]
Sep 11 18:44:51.701714 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:44:51.701684    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a7c92f_6ba4_42bb_b948_c13ae8ad0a13.slice/crio-0f96732cdeaabec4a13d364f1f01c7007a2376f778707c641d720063acb8aa01 WatchSource:0}: Error finding container 0f96732cdeaabec4a13d364f1f01c7007a2376f778707c641d720063acb8aa01: Status 404 returned error can't find the container with id 0f96732cdeaabec4a13d364f1f01c7007a2376f778707c641d720063acb8aa01
Sep 11 18:44:52.538903 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:52.538868    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz" event={"ID":"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13","Type":"ContainerStarted","Data":"f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d"}
Sep 11 18:44:52.539312 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:52.538923    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz" event={"ID":"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13","Type":"ContainerStarted","Data":"0f96732cdeaabec4a13d364f1f01c7007a2376f778707c641d720063acb8aa01"}
Sep 11 18:44:52.539312 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:44:52.538997    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:45:00.134868 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.134809    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz" podStartSLOduration=9.134790902 podStartE2EDuration="9.134790902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:44:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:44:52.557528534 +0000 UTC m=+1755.592745204" watchObservedRunningTime="2026-09-11 18:45:00.134790902 +0000 UTC m=+1763.170007610"
Sep 11 18:45:00.137066 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.137036    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4"]
Sep 11 18:45:00.139973 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.139952    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv"]
Sep 11 18:45:00.140093 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.140068    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4"
Sep 11 18:45:00.142615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.142592    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:45:00.142752 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.142673    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx"]
Sep 11 18:45:00.142838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.142821    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv"
Sep 11 18:45:00.144938 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.144922    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt"]
Sep 11 18:45:00.145049 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.145034    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx"
Sep 11 18:45:00.147253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.147198    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt"
Sep 11 18:45:00.151709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.151693    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4"
Sep 11 18:45:00.164666 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.160732    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4"]
Sep 11 18:45:00.164666 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.160795    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx"]
Sep 11 18:45:00.167061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.167025    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt"]
Sep 11 18:45:00.192597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.192570    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n"]
Sep 11 18:45:00.200258 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.200235    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n"]
Sep 11 18:45:00.200964 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.200946    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n"
Sep 11 18:45:00.203658 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.203538    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv"]
Sep 11 18:45:00.210126 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.210109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv"
Sep 11 18:45:00.219384 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.219365    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx"
Sep 11 18:45:00.265253 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.259107    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt"
Sep 11 18:45:00.282702 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.278568    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n"
Sep 11 18:45:00.477010 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.476984    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4"]
Sep 11 18:45:00.478312 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:45:00.478284    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dec5a0b_c059_4e29_9d3d_5f1d28737c4d.slice/crio-a9243288adfe98264087b80f522b69dfb08a19b2399289d185f5b856dc9df668 WatchSource:0}: Error finding container a9243288adfe98264087b80f522b69dfb08a19b2399289d185f5b856dc9df668: Status 404 returned error can't find the container with id a9243288adfe98264087b80f522b69dfb08a19b2399289d185f5b856dc9df668
Sep 11 18:45:00.518553 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.518512    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx"]
Sep 11 18:45:00.519813 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:45:00.519789    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d9e4fc_4fd6_4afa_a89b_0f4a37d8ac02.slice/crio-529ea4940618dd3c1b724d444df65e1247eb2b7695a54fe2c9aaf43a0eb63862 WatchSource:0}: Error finding container 529ea4940618dd3c1b724d444df65e1247eb2b7695a54fe2c9aaf43a0eb63862: Status 404 returned error can't find the container with id 529ea4940618dd3c1b724d444df65e1247eb2b7695a54fe2c9aaf43a0eb63862
Sep 11 18:45:00.521250 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.521230    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv"]
Sep 11 18:45:00.522894 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:45:00.522873    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98fb8bec_70de_40b6_b9ba_48564b3d0773.slice/crio-3ddab9fdc998edb12c5c9ca692ef10b021f39cb8ddab6c1009139e2e1ab728c3 WatchSource:0}: Error finding container 3ddab9fdc998edb12c5c9ca692ef10b021f39cb8ddab6c1009139e2e1ab728c3: Status 404 returned error can't find the container with id 3ddab9fdc998edb12c5c9ca692ef10b021f39cb8ddab6c1009139e2e1ab728c3
Sep 11 18:45:00.577836 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.577807    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt"]
Sep 11 18:45:00.579883 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:45:00.579858    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9384102_acb4_4ab5_b609_6c8b06daeee6.slice/crio-d4ba07ab44d0decdeb5af28bfd8cc14a972e90d05e28729f8451bec8a0cfe980 WatchSource:0}: Error finding container d4ba07ab44d0decdeb5af28bfd8cc14a972e90d05e28729f8451bec8a0cfe980: Status 404 returned error can't find the container with id d4ba07ab44d0decdeb5af28bfd8cc14a972e90d05e28729f8451bec8a0cfe980
Sep 11 18:45:00.584414 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.584387    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv" event={"ID":"98fb8bec-70de-40b6-b9ba-48564b3d0773","Type":"ContainerStarted","Data":"3ddab9fdc998edb12c5c9ca692ef10b021f39cb8ddab6c1009139e2e1ab728c3"}
Sep 11 18:45:00.584520 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.584479    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n"]
Sep 11 18:45:00.585712 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.585668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx" event={"ID":"19d9e4fc-4fd6-4afa-a89b-0f4a37d8ac02","Type":"ContainerStarted","Data":"529ea4940618dd3c1b724d444df65e1247eb2b7695a54fe2c9aaf43a0eb63862"}
Sep 11 18:45:00.585843 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:45:00.585818    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0f4b6b_cb3f_4cbd_a8cb_0c9693b1fd74.slice/crio-ee5bdc3d43d8b6d2b181a6ccc5a47974773b80725502885d1ea92aaa2c912dab WatchSource:0}: Error finding container ee5bdc3d43d8b6d2b181a6ccc5a47974773b80725502885d1ea92aaa2c912dab: Status 404 returned error can't find the container with id ee5bdc3d43d8b6d2b181a6ccc5a47974773b80725502885d1ea92aaa2c912dab
Sep 11 18:45:00.587046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:00.587030    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4" event={"ID":"4dec5a0b-c059-4e29-9d3d-5f1d28737c4d","Type":"ContainerStarted","Data":"a9243288adfe98264087b80f522b69dfb08a19b2399289d185f5b856dc9df668"}
Sep 11 18:45:01.594544 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.594517    2578 generic.go:356] "Generic (PLEG): container finished" podID="98fb8bec-70de-40b6-b9ba-48564b3d0773" containerID="1e4290d903f8416d14a9ef6020d0580f1e5250416b3e8e9ecfa023b1db02955b" exitCode=0
Sep 11 18:45:01.595346 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.594596    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv" event={"ID":"98fb8bec-70de-40b6-b9ba-48564b3d0773","Type":"ContainerDied","Data":"1e4290d903f8416d14a9ef6020d0580f1e5250416b3e8e9ecfa023b1db02955b"}
Sep 11 18:45:01.596209 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.596193    2578 generic.go:356] "Generic (PLEG): container finished" podID="19d9e4fc-4fd6-4afa-a89b-0f4a37d8ac02" containerID="dd3dd3a53216bf7fb7edc6d71d83ae15ae01c2be40dca0b217909f2a48ffcbb5" exitCode=0
Sep 11 18:45:01.596302 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.596260    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx" event={"ID":"19d9e4fc-4fd6-4afa-a89b-0f4a37d8ac02","Type":"ContainerDied","Data":"dd3dd3a53216bf7fb7edc6d71d83ae15ae01c2be40dca0b217909f2a48ffcbb5"}
Sep 11 18:45:01.597836 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.597813    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4" event={"ID":"4dec5a0b-c059-4e29-9d3d-5f1d28737c4d","Type":"ContainerStarted","Data":"2d4ee33ddf8599aa56cb2b983d51e124a7044fb1035c2a06d09484625d1beab8"}
Sep 11 18:45:01.599428 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.599403    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt" event={"ID":"a9384102-acb4-4ab5-b609-6c8b06daeee6","Type":"ContainerStarted","Data":"8e2698e4ce71a2d48bb5c6d44574ee46150f7b5aceced359a402f09b8d0af649"}
Sep 11 18:45:01.599516 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.599444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt" event={"ID":"a9384102-acb4-4ab5-b609-6c8b06daeee6","Type":"ContainerStarted","Data":"d4ba07ab44d0decdeb5af28bfd8cc14a972e90d05e28729f8451bec8a0cfe980"}
Sep 11 18:45:01.600951 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.600931    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n" event={"ID":"bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74","Type":"ContainerStarted","Data":"b5f62aa1084349647bbb7a63119f7717ff8951eb6c758ddc87d678d37d60a8ff"}
Sep 11 18:45:01.601041 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.600971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n" event={"ID":"bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74","Type":"ContainerStarted","Data":"ee5bdc3d43d8b6d2b181a6ccc5a47974773b80725502885d1ea92aaa2c912dab"}
Sep 11 18:45:01.640152 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.640107    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n" podStartSLOduration=1.640092074 podStartE2EDuration="1.640092074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:01.638006737 +0000 UTC m=+1764.673223406" watchObservedRunningTime="2026-09-11 18:45:01.640092074 +0000 UTC m=+1764.675308741"
Sep 11 18:45:01.652002 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.651830    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4" podStartSLOduration=1.651812386 podStartE2EDuration="1.651812386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:01.651375329 +0000 UTC m=+1764.686592012" watchObservedRunningTime="2026-09-11 18:45:01.651812386 +0000 UTC m=+1764.687029057"
Sep 11 18:45:01.673042 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:01.672997    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt" podStartSLOduration=1.672983194 podStartE2EDuration="1.672983194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:01.671287259 +0000 UTC m=+1764.706503928" watchObservedRunningTime="2026-09-11 18:45:01.672983194 +0000 UTC m=+1764.708199863"
Sep 11 18:45:02.608098 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:02.608064    2578 generic.go:356] "Generic (PLEG): container finished" podID="4dec5a0b-c059-4e29-9d3d-5f1d28737c4d" containerID="2d4ee33ddf8599aa56cb2b983d51e124a7044fb1035c2a06d09484625d1beab8" exitCode=0
Sep 11 18:45:02.608607 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:02.608153    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4" event={"ID":"4dec5a0b-c059-4e29-9d3d-5f1d28737c4d","Type":"ContainerDied","Data":"2d4ee33ddf8599aa56cb2b983d51e124a7044fb1035c2a06d09484625d1beab8"}
Sep 11 18:45:02.609756 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:02.609735    2578 generic.go:356] "Generic (PLEG): container finished" podID="a9384102-acb4-4ab5-b609-6c8b06daeee6" containerID="8e2698e4ce71a2d48bb5c6d44574ee46150f7b5aceced359a402f09b8d0af649" exitCode=0
Sep 11 18:45:02.609864 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:02.609764    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt" event={"ID":"a9384102-acb4-4ab5-b609-6c8b06daeee6","Type":"ContainerDied","Data":"8e2698e4ce71a2d48bb5c6d44574ee46150f7b5aceced359a402f09b8d0af649"}
Sep 11 18:45:02.611231 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:02.611208    2578 generic.go:356] "Generic (PLEG): container finished" podID="bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74" containerID="b5f62aa1084349647bbb7a63119f7717ff8951eb6c758ddc87d678d37d60a8ff" exitCode=0
Sep 11 18:45:02.611319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:02.611292    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n" event={"ID":"bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74","Type":"ContainerDied","Data":"b5f62aa1084349647bbb7a63119f7717ff8951eb6c758ddc87d678d37d60a8ff"}
Sep 11 18:45:02.785350 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:02.785327    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv"
Sep 11 18:45:02.788091 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:02.788070    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx"
Sep 11 18:45:03.546834 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.546808    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:45:03.618279 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.618247    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv"
Sep 11 18:45:03.618279 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.618262    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv" event={"ID":"98fb8bec-70de-40b6-b9ba-48564b3d0773","Type":"ContainerDied","Data":"3ddab9fdc998edb12c5c9ca692ef10b021f39cb8ddab6c1009139e2e1ab728c3"}
Sep 11 18:45:03.618815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.618308    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ddab9fdc998edb12c5c9ca692ef10b021f39cb8ddab6c1009139e2e1ab728c3"
Sep 11 18:45:03.619873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.619853    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx"
Sep 11 18:45:03.620004 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.619896    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx" event={"ID":"19d9e4fc-4fd6-4afa-a89b-0f4a37d8ac02","Type":"ContainerDied","Data":"529ea4940618dd3c1b724d444df65e1247eb2b7695a54fe2c9aaf43a0eb63862"}
Sep 11 18:45:03.620004 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.619935    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="529ea4940618dd3c1b724d444df65e1247eb2b7695a54fe2c9aaf43a0eb63862"
Sep 11 18:45:03.769857 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.769827    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4"
Sep 11 18:45:03.794762 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.794736    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n"
Sep 11 18:45:03.824761 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:03.824736    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt"
Sep 11 18:45:04.626823 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:04.626793    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4"
Sep 11 18:45:04.627298 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:04.626818    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4" event={"ID":"4dec5a0b-c059-4e29-9d3d-5f1d28737c4d","Type":"ContainerDied","Data":"a9243288adfe98264087b80f522b69dfb08a19b2399289d185f5b856dc9df668"}
Sep 11 18:45:04.627298 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:04.626863    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9243288adfe98264087b80f522b69dfb08a19b2399289d185f5b856dc9df668"
Sep 11 18:45:04.628026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:04.628008    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt"
Sep 11 18:45:04.628138 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:04.628021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819205-7vtwt" event={"ID":"a9384102-acb4-4ab5-b609-6c8b06daeee6","Type":"ContainerDied","Data":"d4ba07ab44d0decdeb5af28bfd8cc14a972e90d05e28729f8451bec8a0cfe980"}
Sep 11 18:45:04.628138 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:04.628059    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4ba07ab44d0decdeb5af28bfd8cc14a972e90d05e28729f8451bec8a0cfe980"
Sep 11 18:45:04.629385 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:04.629366    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n"
Sep 11 18:45:04.629493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:04.629383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n" event={"ID":"bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74","Type":"ContainerDied","Data":"ee5bdc3d43d8b6d2b181a6ccc5a47974773b80725502885d1ea92aaa2c912dab"}
Sep 11 18:45:04.629493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:04.629421    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee5bdc3d43d8b6d2b181a6ccc5a47974773b80725502885d1ea92aaa2c912dab"
Sep 11 18:45:18.248306 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248269    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"]
Sep 11 18:45:18.248720 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248694    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9384102-acb4-4ab5-b609-6c8b06daeee6" containerName="cleanup"
Sep 11 18:45:18.248720 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248707    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9384102-acb4-4ab5-b609-6c8b06daeee6" containerName="cleanup"
Sep 11 18:45:18.248720 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248715    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98fb8bec-70de-40b6-b9ba-48564b3d0773" containerName="cleanup"
Sep 11 18:45:18.248830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248720    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="98fb8bec-70de-40b6-b9ba-48564b3d0773" containerName="cleanup"
Sep 11 18:45:18.248830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248734    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dec5a0b-c059-4e29-9d3d-5f1d28737c4d" containerName="cleanup"
Sep 11 18:45:18.248830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248739    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dec5a0b-c059-4e29-9d3d-5f1d28737c4d" containerName="cleanup"
Sep 11 18:45:18.248830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248745    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74" containerName="cleanup"
Sep 11 18:45:18.248830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248750    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74" containerName="cleanup"
Sep 11 18:45:18.248830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248758    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d9e4fc-4fd6-4afa-a89b-0f4a37d8ac02" containerName="cleanup"
Sep 11 18:45:18.248830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248763    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d9e4fc-4fd6-4afa-a89b-0f4a37d8ac02" containerName="cleanup"
Sep 11 18:45:18.248830 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248831    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74" containerName="cleanup"
Sep 11 18:45:18.249104 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248841    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9384102-acb4-4ab5-b609-6c8b06daeee6" containerName="cleanup"
Sep 11 18:45:18.249104 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248847    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dec5a0b-c059-4e29-9d3d-5f1d28737c4d" containerName="cleanup"
Sep 11 18:45:18.249104 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248853    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d9e4fc-4fd6-4afa-a89b-0f4a37d8ac02" containerName="cleanup"
Sep 11 18:45:18.249104 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.248860    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="98fb8bec-70de-40b6-b9ba-48564b3d0773" containerName="cleanup"
Sep 11 18:45:18.251018 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.251001    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"
Sep 11 18:45:18.253570 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.253550    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:45:18.253666 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.253584    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:45:18.261874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.261845    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"]
Sep 11 18:45:18.293052 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.293012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50cfbd06-c73c-4731-a713-0bf27239e5c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-883e85-bqxp4\" (UID: \"50cfbd06-c73c-4731-a713-0bf27239e5c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"
Sep 11 18:45:18.394640 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.394605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50cfbd06-c73c-4731-a713-0bf27239e5c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-883e85-bqxp4\" (UID: \"50cfbd06-c73c-4731-a713-0bf27239e5c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"
Sep 11 18:45:18.395309 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.395284    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50cfbd06-c73c-4731-a713-0bf27239e5c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-883e85-bqxp4\" (UID: \"50cfbd06-c73c-4731-a713-0bf27239e5c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"
Sep 11 18:45:18.565232 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.565135    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"
Sep 11 18:45:18.842641 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:18.842598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"]
Sep 11 18:45:18.844432 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:45:18.844401    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50cfbd06_c73c_4731_a713_0bf27239e5c3.slice/crio-71733842448f10e740042f16aeb63a98430e13c4eb45b4044bcebb44bfd1a18f WatchSource:0}: Error finding container 71733842448f10e740042f16aeb63a98430e13c4eb45b4044bcebb44bfd1a18f: Status 404 returned error can't find the container with id 71733842448f10e740042f16aeb63a98430e13c4eb45b4044bcebb44bfd1a18f
Sep 11 18:45:19.726652 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:19.726615    2578 generic.go:356] "Generic (PLEG): container finished" podID="50cfbd06-c73c-4731-a713-0bf27239e5c3" containerID="8073062dbdbf23b704390c632d1307d31da4a19eb91214a6c8e66625b46b460f" exitCode=0
Sep 11 18:45:19.727159 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:19.726713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4" event={"ID":"50cfbd06-c73c-4731-a713-0bf27239e5c3","Type":"ContainerDied","Data":"8073062dbdbf23b704390c632d1307d31da4a19eb91214a6c8e66625b46b460f"}
Sep 11 18:45:19.727159 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:19.726770    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4" event={"ID":"50cfbd06-c73c-4731-a713-0bf27239e5c3","Type":"ContainerStarted","Data":"71733842448f10e740042f16aeb63a98430e13c4eb45b4044bcebb44bfd1a18f"}
Sep 11 18:45:20.733925 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.733844    2578 generic.go:356] "Generic (PLEG): container finished" podID="8030c458-6d3d-44ed-97ca-a92701958f74" containerID="f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4" exitCode=0
Sep 11 18:45:20.734343 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.733917    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" event={"ID":"8030c458-6d3d-44ed-97ca-a92701958f74","Type":"ContainerDied","Data":"f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4"}
Sep 11 18:45:20.734805 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.734433    2578 scope.go:117] "RemoveContainer" containerID="f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4"
Sep 11 18:45:20.735722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.735663    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:45:20.735851 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.735777    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:45:20.735851 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.735823    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:45:20.881643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.881338    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"
Sep 11 18:45:20.922118 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.922082    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50cfbd06-c73c-4731-a713-0bf27239e5c3-maas-api-service-ca\") pod \"50cfbd06-c73c-4731-a713-0bf27239e5c3\" (UID: \"50cfbd06-c73c-4731-a713-0bf27239e5c3\") "
Sep 11 18:45:20.922426 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.922403    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50cfbd06-c73c-4731-a713-0bf27239e5c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50cfbd06-c73c-4731-a713-0bf27239e5c3" (UID: "50cfbd06-c73c-4731-a713-0bf27239e5c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:45:20.922539 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:20.922518    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50cfbd06-c73c-4731-a713-0bf27239e5c3-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:21.746919 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:21.745856    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" event={"ID":"8030c458-6d3d-44ed-97ca-a92701958f74","Type":"ContainerStarted","Data":"1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805"}
Sep 11 18:45:21.750099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:21.749445    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4" event={"ID":"50cfbd06-c73c-4731-a713-0bf27239e5c3","Type":"ContainerDied","Data":"71733842448f10e740042f16aeb63a98430e13c4eb45b4044bcebb44bfd1a18f"}
Sep 11 18:45:21.750099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:21.749503    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71733842448f10e740042f16aeb63a98430e13c4eb45b4044bcebb44bfd1a18f"
Sep 11 18:45:21.750099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:21.749563    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"
Sep 11 18:45:22.300379 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:22.300341    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:45:22.300568 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:22.300545    2578 patch_prober.go:28] interesting pod/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 11 18:45:22.300620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:22.300597    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 11 18:45:23.300621 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:23.300585    2578 patch_prober.go:28] interesting pod/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 11 18:45:23.301047 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:23.300650    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 11 18:45:24.301429 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:24.301389    2578 patch_prober.go:28] interesting pod/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 11 18:45:24.301854 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:24.301463    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 11 18:45:25.300929 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:25.300894    2578 patch_prober.go:28] interesting pod/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 11 18:45:25.301099 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:25.300958    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 11 18:45:26.300888 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:26.300852    2578 patch_prober.go:28] interesting pod/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 11 18:45:26.301273 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:26.300904    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 11 18:45:27.304540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:27.304504    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:45:27.304930 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:27.304871    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:45:27.305584 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:27.305564    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:45:34.004112 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:34.004023    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"]
Sep 11 18:45:34.006347 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:34.006323    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-883e85-bqxp4"]
Sep 11 18:45:35.410515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:35.410486    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50cfbd06-c73c-4731-a713-0bf27239e5c3" path="/var/lib/kubelet/pods/50cfbd06-c73c-4731-a713-0bf27239e5c3/volumes"
Sep 11 18:45:37.520759 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:37.520732    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:45:37.524911 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:37.524888    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:45:37.529553 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:37.529533    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:45:37.533668 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:37.533651    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:45:37.747135 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:37.747101    2578 scope.go:117] "RemoveContainer" containerID="ad2856119db5240434fbba42ff0b8f60c808504e86799ed6ee2001569d21ac79"
Sep 11 18:45:40.487395 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.487357    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"]
Sep 11 18:45:40.487909 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.487664    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy" containerID="cri-o://d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76" gracePeriod=30
Sep 11 18:45:40.826261 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.826173    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"]
Sep 11 18:45:40.826595 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.826582    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50cfbd06-c73c-4731-a713-0bf27239e5c3" containerName="revoke-keys"
Sep 11 18:45:40.826641 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.826596    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50cfbd06-c73c-4731-a713-0bf27239e5c3" containerName="revoke-keys"
Sep 11 18:45:40.826710 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.826663    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="50cfbd06-c73c-4731-a713-0bf27239e5c3" containerName="revoke-keys"
Sep 11 18:45:40.829046 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.829030    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"
Sep 11 18:45:40.831290 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.831267    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:45:40.831386 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.831329    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:45:40.837904 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.837882    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"]
Sep 11 18:45:40.943888 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:40.943853    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554f8584-5c4b-4917-9d81-475bc1122d97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg\" (UID: \"554f8584-5c4b-4917-9d81-475bc1122d97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"
Sep 11 18:45:41.044793 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:41.044760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554f8584-5c4b-4917-9d81-475bc1122d97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg\" (UID: \"554f8584-5c4b-4917-9d81-475bc1122d97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"
Sep 11 18:45:41.045475 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:41.045448    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554f8584-5c4b-4917-9d81-475bc1122d97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg\" (UID: \"554f8584-5c4b-4917-9d81-475bc1122d97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"
Sep 11 18:45:41.140084 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:41.140012    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"
Sep 11 18:45:41.411227 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:41.411198    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"]
Sep 11 18:45:41.412785 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:45:41.412756    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554f8584_5c4b_4917_9d81_475bc1122d97.slice/crio-b9786d2994c5e7256331fdcee9356324e2a3a7a01a8a55bd3460e4653a360aa8 WatchSource:0}: Error finding container b9786d2994c5e7256331fdcee9356324e2a3a7a01a8a55bd3460e4653a360aa8: Status 404 returned error can't find the container with id b9786d2994c5e7256331fdcee9356324e2a3a7a01a8a55bd3460e4653a360aa8
Sep 11 18:45:41.864493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:41.864452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg" event={"ID":"554f8584-5c4b-4917-9d81-475bc1122d97","Type":"ContainerStarted","Data":"df696a7e26dd47a4c0ea89265cadd53e65ce50e6748246f69ff800b563525399"}
Sep 11 18:45:41.865240 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:41.864512    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg" event={"ID":"554f8584-5c4b-4917-9d81-475bc1122d97","Type":"ContainerStarted","Data":"b9786d2994c5e7256331fdcee9356324e2a3a7a01a8a55bd3460e4653a360aa8"}
Sep 11 18:45:41.881794 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:41.881751    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg" podStartSLOduration=1.8817368939999999 podStartE2EDuration="1.881736894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:41.880474014 +0000 UTC m=+1804.915690682" watchObservedRunningTime="2026-09-11 18:45:41.881736894 +0000 UTC m=+1804.916953563"
Sep 11 18:45:42.190569 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:42.190530    2578 patch_prober.go:28] interesting pod/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:45:42.190705 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:42.190584    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:45:42.871947 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:42.871915    2578 generic.go:356] "Generic (PLEG): container finished" podID="554f8584-5c4b-4917-9d81-475bc1122d97" containerID="df696a7e26dd47a4c0ea89265cadd53e65ce50e6748246f69ff800b563525399" exitCode=0
Sep 11 18:45:42.872341 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:42.871951    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg" event={"ID":"554f8584-5c4b-4917-9d81-475bc1122d97","Type":"ContainerDied","Data":"df696a7e26dd47a4c0ea89265cadd53e65ce50e6748246f69ff800b563525399"}
Sep 11 18:45:44.016462 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:44.016436    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"
Sep 11 18:45:44.178632 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:44.178606    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554f8584-5c4b-4917-9d81-475bc1122d97-maas-api-service-ca\") pod \"554f8584-5c4b-4917-9d81-475bc1122d97\" (UID: \"554f8584-5c4b-4917-9d81-475bc1122d97\") "
Sep 11 18:45:44.178976 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:44.178952    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/554f8584-5c4b-4917-9d81-475bc1122d97-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "554f8584-5c4b-4917-9d81-475bc1122d97" (UID: "554f8584-5c4b-4917-9d81-475bc1122d97"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:45:44.280163 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:44.280136    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554f8584-5c4b-4917-9d81-475bc1122d97-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:44.884620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:44.884586    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"
Sep 11 18:45:44.884620 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:44.884597    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg" event={"ID":"554f8584-5c4b-4917-9d81-475bc1122d97","Type":"ContainerDied","Data":"b9786d2994c5e7256331fdcee9356324e2a3a7a01a8a55bd3460e4653a360aa8"}
Sep 11 18:45:44.884838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:44.884638    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9786d2994c5e7256331fdcee9356324e2a3a7a01a8a55bd3460e4653a360aa8"
Sep 11 18:45:45.734461 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.734434    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:45:45.791167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791082    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-envoy\") pod \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") "
Sep 11 18:45:45.791167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791122    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-data\") pod \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") "
Sep 11 18:45:45.791167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791148    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-podinfo\") pod \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") "
Sep 11 18:45:45.791443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791180    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-socket\") pod \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") "
Sep 11 18:45:45.791443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791209    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-credential-socket\") pod \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") "
Sep 11 18:45:45.791443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791248    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-certs\") pod \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") "
Sep 11 18:45:45.791443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791272    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc4j6\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-kube-api-access-wc4j6\") pod \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") "
Sep 11 18:45:45.791443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791358    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istiod-ca-cert\") pod \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") "
Sep 11 18:45:45.791443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791369    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-data" (OuterVolumeSpecName: "istio-data") pod "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" (UID: "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:45.791443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791410    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-token\") pod \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\" (UID: \"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a\") "
Sep 11 18:45:45.791819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791533    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" (UID: "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:45.791819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791553    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" (UID: "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:45.791819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791779    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:45.791819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791799    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:45.791819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791812    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:45.792023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791837    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" (UID: "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:45:45.792023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.791884    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" (UID: "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:45.793868 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.793837    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" (UID: "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:45.794001 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.793866    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-kube-api-access-wc4j6" (OuterVolumeSpecName: "kube-api-access-wc4j6") pod "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" (UID: "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a"). InnerVolumeSpecName "kube-api-access-wc4j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:45:45.794001 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.793902    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" (UID: "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:45:45.794001 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.793932    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-token" (OuterVolumeSpecName: "istio-token") pod "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" (UID: "db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:45:45.891786 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.891750    2578 generic.go:356] "Generic (PLEG): container finished" podID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerID="d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76" exitCode=0
Sep 11 18:45:45.891998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.891804    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"
Sep 11 18:45:45.891998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.891836    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" event={"ID":"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a","Type":"ContainerDied","Data":"d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76"}
Sep 11 18:45:45.891998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.891902    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq" event={"ID":"db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a","Type":"ContainerDied","Data":"025665dea957967bcf9b7645c7056c98f0dae2bea4541300114e969b2171d253"}
Sep 11 18:45:45.891998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.891925    2578 scope.go:117] "RemoveContainer" containerID="d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76"
Sep 11 18:45:45.892374 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.892352    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:45.892473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.892378    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:45.892473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.892387    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:45.892473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.892395    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:45.892473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.892402    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:45.892473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.892410    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc4j6\" (UniqueName: \"kubernetes.io/projected/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a-kube-api-access-wc4j6\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:45.901790 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.901772    2578 scope.go:117] "RemoveContainer" containerID="921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700"
Sep 11 18:45:45.910714 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.910693    2578 scope.go:117] "RemoveContainer" containerID="d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76"
Sep 11 18:45:45.910978 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:45:45.910960    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76\": container with ID starting with d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76 not found: ID does not exist" containerID="d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76"
Sep 11 18:45:45.911034 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.910987    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76"} err="failed to get container status \"d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76\": rpc error: code = NotFound desc = could not find container \"d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76\": container with ID starting with d93a68e7afa6989ef46a1b0c3e9a3c568d5b17ed77d43dbc2ff5fe050f78ca76 not found: ID does not exist"
Sep 11 18:45:45.911034 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.911005    2578 scope.go:117] "RemoveContainer" containerID="921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700"
Sep 11 18:45:45.911256 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:45:45.911237    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700\": container with ID starting with 921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700 not found: ID does not exist" containerID="921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700"
Sep 11 18:45:45.911310 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.911265    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700"} err="failed to get container status \"921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700\": rpc error: code = NotFound desc = could not find container \"921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700\": container with ID starting with 921eac0197ee9b14b3104b796055e1921c84ab1eaaf44744237875b993663700 not found: ID does not exist"
Sep 11 18:45:45.916772 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.916749    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"]
Sep 11 18:45:45.924432 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:45.924411    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-883e85-openshift-default-8446658c86-mkmlq"]
Sep 11 18:45:47.410326 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:47.410294    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" path="/var/lib/kubelet/pods/db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a/volumes"
Sep 11 18:45:56.495287 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.495255    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"]
Sep 11 18:45:56.495751 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.495495    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz" podUID="09a7c92f-6ba4-42bb-b948-c13ae8ad0a13" containerName="payload-pre-processing" containerID="cri-o://f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d" gracePeriod=30
Sep 11 18:45:56.746301 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.746246    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:45:56.757119 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.757095    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"]
Sep 11 18:45:56.760570 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.760549    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5cbc66-r5fdg"]
Sep 11 18:45:56.793491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.793454    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvgpl\" (UniqueName: \"kubernetes.io/projected/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-kube-api-access-pvgpl\") pod \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\" (UID: \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\") "
Sep 11 18:45:56.793491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.793493    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-plugins-config-volume\") pod \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\" (UID: \"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13\") "
Sep 11 18:45:56.793899 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.793873    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09a7c92f-6ba4-42bb-b948-c13ae8ad0a13" (UID: "09a7c92f-6ba4-42bb-b948-c13ae8ad0a13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:45:56.795526 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.795501    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-kube-api-access-pvgpl" (OuterVolumeSpecName: "kube-api-access-pvgpl") pod "09a7c92f-6ba4-42bb-b948-c13ae8ad0a13" (UID: "09a7c92f-6ba4-42bb-b948-c13ae8ad0a13"). InnerVolumeSpecName "kube-api-access-pvgpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:45:56.895176 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.895141    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvgpl\" (UniqueName: \"kubernetes.io/projected/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-kube-api-access-pvgpl\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:56.895176 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.895172    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:45:56.968987 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.968926    2578 generic.go:356] "Generic (PLEG): container finished" podID="09a7c92f-6ba4-42bb-b948-c13ae8ad0a13" containerID="f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d" exitCode=0
Sep 11 18:45:56.968987 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.968982    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"
Sep 11 18:45:56.969257 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.968981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz" event={"ID":"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13","Type":"ContainerDied","Data":"f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d"}
Sep 11 18:45:56.969257 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.969052    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz" event={"ID":"09a7c92f-6ba4-42bb-b948-c13ae8ad0a13","Type":"ContainerDied","Data":"0f96732cdeaabec4a13d364f1f01c7007a2376f778707c641d720063acb8aa01"}
Sep 11 18:45:56.969257 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.969084    2578 scope.go:117] "RemoveContainer" containerID="f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d"
Sep 11 18:45:56.973695 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.973660    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"]
Sep 11 18:45:56.974404 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.974387    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy"
Sep 11 18:45:56.974513 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.974503    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy"
Sep 11 18:45:56.974609 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.974590    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a7c92f-6ba4-42bb-b948-c13ae8ad0a13" containerName="payload-pre-processing"
Sep 11 18:45:56.974609 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.974607    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a7c92f-6ba4-42bb-b948-c13ae8ad0a13" containerName="payload-pre-processing"
Sep 11 18:45:56.974838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.974621    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554f8584-5c4b-4917-9d81-475bc1122d97" containerName="revoke-keys"
Sep 11 18:45:56.974838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.974629    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="554f8584-5c4b-4917-9d81-475bc1122d97" containerName="revoke-keys"
Sep 11 18:45:56.974838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.974753    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy"
Sep 11 18:45:56.974838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.974767    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="554f8584-5c4b-4917-9d81-475bc1122d97" containerName="revoke-keys"
Sep 11 18:45:56.974838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.974780    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="09a7c92f-6ba4-42bb-b948-c13ae8ad0a13" containerName="payload-pre-processing"
Sep 11 18:45:56.977319 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.977297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"
Sep 11 18:45:56.979815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.979795    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:45:56.979930 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.979794    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:45:56.984075 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.983814    2578 scope.go:117] "RemoveContainer" containerID="f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d"
Sep 11 18:45:56.984517 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:45:56.984495    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d\": container with ID starting with f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d not found: ID does not exist" containerID="f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d"
Sep 11 18:45:56.984669 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.984647    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d"} err="failed to get container status \"f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d\": rpc error: code = NotFound desc = could not find container \"f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d\": container with ID starting with f54b932632f566710b975baf988800b36940cf68952ab0caa9aa6e823bd8832d not found: ID does not exist"
Sep 11 18:45:56.988039 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.987995    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"]
Sep 11 18:45:56.996390 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:56.996332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856d1224-5cc1-41f6-bd86-d875b5346eb5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd\" (UID: \"856d1224-5cc1-41f6-bd86-d875b5346eb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"
Sep 11 18:45:57.014422 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.014396    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"]
Sep 11 18:45:57.018049 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.018028    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5cbc66-5cdc4cbdc8-wvfwz"]
Sep 11 18:45:57.097576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.097525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856d1224-5cc1-41f6-bd86-d875b5346eb5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd\" (UID: \"856d1224-5cc1-41f6-bd86-d875b5346eb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"
Sep 11 18:45:57.098270 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.098252    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856d1224-5cc1-41f6-bd86-d875b5346eb5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd\" (UID: \"856d1224-5cc1-41f6-bd86-d875b5346eb5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"
Sep 11 18:45:57.292670 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.292587    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"
Sep 11 18:45:57.410446 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.410397    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09a7c92f-6ba4-42bb-b948-c13ae8ad0a13" path="/var/lib/kubelet/pods/09a7c92f-6ba4-42bb-b948-c13ae8ad0a13/volumes"
Sep 11 18:45:57.410787 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.410774    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="554f8584-5c4b-4917-9d81-475bc1122d97" path="/var/lib/kubelet/pods/554f8584-5c4b-4917-9d81-475bc1122d97/volumes"
Sep 11 18:45:57.562666 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.562641    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"]
Sep 11 18:45:57.564760 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:45:57.564726    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856d1224_5cc1_41f6_bd86_d875b5346eb5.slice/crio-ed0b729d93ddacdc5ea64ca9d5c5e79ee796f7d08f829a41846de226ac5ff939 WatchSource:0}: Error finding container ed0b729d93ddacdc5ea64ca9d5c5e79ee796f7d08f829a41846de226ac5ff939: Status 404 returned error can't find the container with id ed0b729d93ddacdc5ea64ca9d5c5e79ee796f7d08f829a41846de226ac5ff939
Sep 11 18:45:57.976398 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.976358    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd" event={"ID":"856d1224-5cc1-41f6-bd86-d875b5346eb5","Type":"ContainerStarted","Data":"2994dfc0ac950521f9852e62ccc2c4f87b6354aa5f3096e23d59bcef062434fe"}
Sep 11 18:45:57.976586 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.976438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd" event={"ID":"856d1224-5cc1-41f6-bd86-d875b5346eb5","Type":"ContainerStarted","Data":"ed0b729d93ddacdc5ea64ca9d5c5e79ee796f7d08f829a41846de226ac5ff939"}
Sep 11 18:45:57.994574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:57.994529    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd" podStartSLOduration=1.994513785 podStartE2EDuration="1.994513785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:57.992276179 +0000 UTC m=+1821.027492847" watchObservedRunningTime="2026-09-11 18:45:57.994513785 +0000 UTC m=+1821.029730497"
Sep 11 18:45:58.984168 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:58.984135    2578 generic.go:356] "Generic (PLEG): container finished" podID="856d1224-5cc1-41f6-bd86-d875b5346eb5" containerID="2994dfc0ac950521f9852e62ccc2c4f87b6354aa5f3096e23d59bcef062434fe" exitCode=0
Sep 11 18:45:58.984550 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:45:58.984220    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd" event={"ID":"856d1224-5cc1-41f6-bd86-d875b5346eb5","Type":"ContainerDied","Data":"2994dfc0ac950521f9852e62ccc2c4f87b6354aa5f3096e23d59bcef062434fe"}
Sep 11 18:46:00.125425 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:00.125386    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"
Sep 11 18:46:00.231478 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:00.231442    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856d1224-5cc1-41f6-bd86-d875b5346eb5-maas-api-service-ca\") pod \"856d1224-5cc1-41f6-bd86-d875b5346eb5\" (UID: \"856d1224-5cc1-41f6-bd86-d875b5346eb5\") "
Sep 11 18:46:00.231843 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:00.231820    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/856d1224-5cc1-41f6-bd86-d875b5346eb5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "856d1224-5cc1-41f6-bd86-d875b5346eb5" (UID: "856d1224-5cc1-41f6-bd86-d875b5346eb5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:46:00.231946 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:00.231934    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856d1224-5cc1-41f6-bd86-d875b5346eb5-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:00.996615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:00.996576    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd" event={"ID":"856d1224-5cc1-41f6-bd86-d875b5346eb5","Type":"ContainerDied","Data":"ed0b729d93ddacdc5ea64ca9d5c5e79ee796f7d08f829a41846de226ac5ff939"}
Sep 11 18:46:00.996823 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:00.996632    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed0b729d93ddacdc5ea64ca9d5c5e79ee796f7d08f829a41846de226ac5ff939"
Sep 11 18:46:00.996823 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:00.996594    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"
Sep 11 18:46:03.208729 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.208693    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"]
Sep 11 18:46:03.209129 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.208934    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy" containerID="cri-o://1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805" gracePeriod=30
Sep 11 18:46:03.559025 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.558945    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"]
Sep 11 18:46:03.559353 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.559341    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy"
Sep 11 18:46:03.559396 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.559355    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy"
Sep 11 18:46:03.559396 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.559366    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="856d1224-5cc1-41f6-bd86-d875b5346eb5" containerName="revoke-keys"
Sep 11 18:46:03.559396 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.559372    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="856d1224-5cc1-41f6-bd86-d875b5346eb5" containerName="revoke-keys"
Sep 11 18:46:03.559493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.559448    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="856d1224-5cc1-41f6-bd86-d875b5346eb5" containerName="revoke-keys"
Sep 11 18:46:03.559493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.559458    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="db6fd2b2-a2a3-479b-a9f6-ccf45a96f95a" containerName="istio-proxy"
Sep 11 18:46:03.561842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.561826    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"
Sep 11 18:46:03.564359 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.564337    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:46:03.564488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.564371    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:46:03.570778 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.570756    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"]
Sep 11 18:46:03.665747 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.665716    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee5842dd-b491-4b78-a58a-c5132f8140e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2\" (UID: \"ee5842dd-b491-4b78-a58a-c5132f8140e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"
Sep 11 18:46:03.766306 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.766272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee5842dd-b491-4b78-a58a-c5132f8140e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2\" (UID: \"ee5842dd-b491-4b78-a58a-c5132f8140e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"
Sep 11 18:46:03.766977 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.766959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee5842dd-b491-4b78-a58a-c5132f8140e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2\" (UID: \"ee5842dd-b491-4b78-a58a-c5132f8140e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"
Sep 11 18:46:03.873004 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:03.872925    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"
Sep 11 18:46:04.144011 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:04.143984    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"]
Sep 11 18:46:04.149408 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:46:04.149298    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5842dd_b491_4b78_a58a_c5132f8140e9.slice/crio-0fccf59a45a39e5d2e98d3f377acf7cd33cd234e7e52bdd8c4940c3571d5fb4a WatchSource:0}: Error finding container 0fccf59a45a39e5d2e98d3f377acf7cd33cd234e7e52bdd8c4940c3571d5fb4a: Status 404 returned error can't find the container with id 0fccf59a45a39e5d2e98d3f377acf7cd33cd234e7e52bdd8c4940c3571d5fb4a
Sep 11 18:46:05.020023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:05.019989    2578 generic.go:356] "Generic (PLEG): container finished" podID="ee5842dd-b491-4b78-a58a-c5132f8140e9" containerID="055c366e15454d4d6e683348dd96b5aabe0ce0fbee44f03c7a65ea614f07c0f7" exitCode=0
Sep 11 18:46:05.020381 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:05.020036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2" event={"ID":"ee5842dd-b491-4b78-a58a-c5132f8140e9","Type":"ContainerDied","Data":"055c366e15454d4d6e683348dd96b5aabe0ce0fbee44f03c7a65ea614f07c0f7"}
Sep 11 18:46:05.020381 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:05.020076    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2" event={"ID":"ee5842dd-b491-4b78-a58a-c5132f8140e9","Type":"ContainerStarted","Data":"0fccf59a45a39e5d2e98d3f377acf7cd33cd234e7e52bdd8c4940c3571d5fb4a"}
Sep 11 18:46:06.170074 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:06.170046    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"
Sep 11 18:46:06.293078 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:06.293002    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee5842dd-b491-4b78-a58a-c5132f8140e9-maas-api-service-ca\") pod \"ee5842dd-b491-4b78-a58a-c5132f8140e9\" (UID: \"ee5842dd-b491-4b78-a58a-c5132f8140e9\") "
Sep 11 18:46:06.293404 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:06.293379    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee5842dd-b491-4b78-a58a-c5132f8140e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ee5842dd-b491-4b78-a58a-c5132f8140e9" (UID: "ee5842dd-b491-4b78-a58a-c5132f8140e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:46:06.394691 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:06.394653    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee5842dd-b491-4b78-a58a-c5132f8140e9-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:07.038088 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:07.038060    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"
Sep 11 18:46:07.038237 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:07.038054    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2" event={"ID":"ee5842dd-b491-4b78-a58a-c5132f8140e9","Type":"ContainerDied","Data":"0fccf59a45a39e5d2e98d3f377acf7cd33cd234e7e52bdd8c4940c3571d5fb4a"}
Sep 11 18:46:07.038237 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:07.038205    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fccf59a45a39e5d2e98d3f377acf7cd33cd234e7e52bdd8c4940c3571d5fb4a"
Sep 11 18:46:08.461206 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.461178    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:46:08.514313 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514232    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-data\") pod \"8030c458-6d3d-44ed-97ca-a92701958f74\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") "
Sep 11 18:46:08.514313 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514278    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8030c458-6d3d-44ed-97ca-a92701958f74-istiod-ca-cert\") pod \"8030c458-6d3d-44ed-97ca-a92701958f74\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") "
Sep 11 18:46:08.514532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514318    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-envoy\") pod \"8030c458-6d3d-44ed-97ca-a92701958f74\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") "
Sep 11 18:46:08.514532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514348    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8030c458-6d3d-44ed-97ca-a92701958f74-istio-podinfo\") pod \"8030c458-6d3d-44ed-97ca-a92701958f74\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") "
Sep 11 18:46:08.514532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514373    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-socket\") pod \"8030c458-6d3d-44ed-97ca-a92701958f74\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") "
Sep 11 18:46:08.514532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514458    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-certs\") pod \"8030c458-6d3d-44ed-97ca-a92701958f74\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") "
Sep 11 18:46:08.514532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514489    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-credential-socket\") pod \"8030c458-6d3d-44ed-97ca-a92701958f74\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") "
Sep 11 18:46:08.514826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514545    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-data" (OuterVolumeSpecName: "istio-data") pod "8030c458-6d3d-44ed-97ca-a92701958f74" (UID: "8030c458-6d3d-44ed-97ca-a92701958f74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:08.514826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514590    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-istio-token\") pod \"8030c458-6d3d-44ed-97ca-a92701958f74\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") "
Sep 11 18:46:08.514826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514640    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpjzg\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-kube-api-access-cpjzg\") pod \"8030c458-6d3d-44ed-97ca-a92701958f74\" (UID: \"8030c458-6d3d-44ed-97ca-a92701958f74\") "
Sep 11 18:46:08.514826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514789    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8030c458-6d3d-44ed-97ca-a92701958f74" (UID: "8030c458-6d3d-44ed-97ca-a92701958f74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:08.514826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514807    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8030c458-6d3d-44ed-97ca-a92701958f74" (UID: "8030c458-6d3d-44ed-97ca-a92701958f74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:08.514826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.514802    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8030c458-6d3d-44ed-97ca-a92701958f74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8030c458-6d3d-44ed-97ca-a92701958f74" (UID: "8030c458-6d3d-44ed-97ca-a92701958f74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:46:08.515123 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.515075    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8030c458-6d3d-44ed-97ca-a92701958f74" (UID: "8030c458-6d3d-44ed-97ca-a92701958f74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:08.515188 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.515128    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:08.515188 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.515149    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8030c458-6d3d-44ed-97ca-a92701958f74-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:08.515188 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.515163    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:08.515188 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.515175    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:08.516619 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.516591    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8030c458-6d3d-44ed-97ca-a92701958f74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8030c458-6d3d-44ed-97ca-a92701958f74" (UID: "8030c458-6d3d-44ed-97ca-a92701958f74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:46:08.516741 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.516718    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-istio-token" (OuterVolumeSpecName: "istio-token") pod "8030c458-6d3d-44ed-97ca-a92701958f74" (UID: "8030c458-6d3d-44ed-97ca-a92701958f74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:46:08.516905 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.516882    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8030c458-6d3d-44ed-97ca-a92701958f74" (UID: "8030c458-6d3d-44ed-97ca-a92701958f74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:08.517049 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.517033    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-kube-api-access-cpjzg" (OuterVolumeSpecName: "kube-api-access-cpjzg") pod "8030c458-6d3d-44ed-97ca-a92701958f74" (UID: "8030c458-6d3d-44ed-97ca-a92701958f74"). InnerVolumeSpecName "kube-api-access-cpjzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:46:08.616016 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.615981    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:08.616016 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.616012    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8030c458-6d3d-44ed-97ca-a92701958f74-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:08.616016 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.616020    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8030c458-6d3d-44ed-97ca-a92701958f74-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:08.616248 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.616030    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:08.616248 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:08.616038    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpjzg\" (UniqueName: \"kubernetes.io/projected/8030c458-6d3d-44ed-97ca-a92701958f74-kube-api-access-cpjzg\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:09.051026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.050994    2578 generic.go:356] "Generic (PLEG): container finished" podID="8030c458-6d3d-44ed-97ca-a92701958f74" containerID="1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805" exitCode=0
Sep 11 18:46:09.051172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.051038    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"
Sep 11 18:46:09.051172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.051075    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" event={"ID":"8030c458-6d3d-44ed-97ca-a92701958f74","Type":"ContainerDied","Data":"1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805"}
Sep 11 18:46:09.051172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.051130    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k" event={"ID":"8030c458-6d3d-44ed-97ca-a92701958f74","Type":"ContainerDied","Data":"e1f9ebad9e5c7caf89754ef6e37fbdfc8713ccdef1613b0e9f77e2eea1ae7e16"}
Sep 11 18:46:09.051172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.051153    2578 scope.go:117] "RemoveContainer" containerID="1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805"
Sep 11 18:46:09.061050 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.061034    2578 scope.go:117] "RemoveContainer" containerID="f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4"
Sep 11 18:46:09.069044 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.069028    2578 scope.go:117] "RemoveContainer" containerID="1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805"
Sep 11 18:46:09.069331 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:46:09.069310    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805\": container with ID starting with 1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805 not found: ID does not exist" containerID="1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805"
Sep 11 18:46:09.069458 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.069336    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805"} err="failed to get container status \"1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805\": rpc error: code = NotFound desc = could not find container \"1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805\": container with ID starting with 1b1039e37292dd6bb9f06ddb00e5906157d9d2eb52e6d3cffc4981151b759805 not found: ID does not exist"
Sep 11 18:46:09.069458 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.069353    2578 scope.go:117] "RemoveContainer" containerID="f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4"
Sep 11 18:46:09.069661 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:46:09.069614    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4\": container with ID starting with f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4 not found: ID does not exist" containerID="f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4"
Sep 11 18:46:09.069661 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.069640    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4"} err="failed to get container status \"f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4\": rpc error: code = NotFound desc = could not find container \"f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4\": container with ID starting with f59f7aed9d3784992ee8beaf71e56d4840c4fa6a1ba1d6470f1bf1d6d8bbe0c4 not found: ID does not exist"
Sep 11 18:46:09.072762 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.072740    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"]
Sep 11 18:46:09.078547 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.078520    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-5cbc66-openshift-default-8cb77c7f6-7hc6k"]
Sep 11 18:46:09.410293 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:09.410267    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" path="/var/lib/kubelet/pods/8030c458-6d3d-44ed-97ca-a92701958f74/volumes"
Sep 11 18:46:14.044359 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.044320    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4"]
Sep 11 18:46:14.046666 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.046637    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-200c07-29819205-28gb4"]
Sep 11 18:46:14.077322 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.077081    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"]
Sep 11 18:46:14.077573 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.077337    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql" podUID="644c1a84-c849-4a1f-8ba5-3c358d956678" containerName="payload-pre-processing" containerID="cri-o://03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa" gracePeriod=30
Sep 11 18:46:14.230758 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.230582    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"]
Sep 11 18:46:14.230962 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.230892    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf" podUID="c194fbca-c1ac-4441-9ab4-1f902110b978" containerName="payload-processing" containerID="cri-o://35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b" gracePeriod=30
Sep 11 18:46:14.239452 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.239209    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"]
Sep 11 18:46:14.240063 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.239566    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb" podUID="640bcbc4-85c5-4895-acbe-f2d5737193d4" containerName="payload-pre-processing" containerID="cri-o://759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b" gracePeriod=30
Sep 11 18:46:14.271794 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.271763    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"]
Sep 11 18:46:14.278794 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.278768    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-200c07-mz5xd"]
Sep 11 18:46:14.378398 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.377147    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:46:14.473234 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.473197    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcd9z\" (UniqueName: \"kubernetes.io/projected/644c1a84-c849-4a1f-8ba5-3c358d956678-kube-api-access-gcd9z\") pod \"644c1a84-c849-4a1f-8ba5-3c358d956678\" (UID: \"644c1a84-c849-4a1f-8ba5-3c358d956678\") "
Sep 11 18:46:14.473384 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.473320    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/644c1a84-c849-4a1f-8ba5-3c358d956678-plugins-config-volume\") pod \"644c1a84-c849-4a1f-8ba5-3c358d956678\" (UID: \"644c1a84-c849-4a1f-8ba5-3c358d956678\") "
Sep 11 18:46:14.473840 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.473808    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/644c1a84-c849-4a1f-8ba5-3c358d956678-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "644c1a84-c849-4a1f-8ba5-3c358d956678" (UID: "644c1a84-c849-4a1f-8ba5-3c358d956678"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:46:14.476371 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.476345    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/644c1a84-c849-4a1f-8ba5-3c358d956678-kube-api-access-gcd9z" (OuterVolumeSpecName: "kube-api-access-gcd9z") pod "644c1a84-c849-4a1f-8ba5-3c358d956678" (UID: "644c1a84-c849-4a1f-8ba5-3c358d956678"). InnerVolumeSpecName "kube-api-access-gcd9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:46:14.514037 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.514017    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:46:14.519585 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.519557    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:46:14.563314 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.563230    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"]
Sep 11 18:46:14.565856 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.565830    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72c98d-6r9v2"]
Sep 11 18:46:14.574873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.574852    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcd9z\" (UniqueName: \"kubernetes.io/projected/644c1a84-c849-4a1f-8ba5-3c358d956678-kube-api-access-gcd9z\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:14.574873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.574873    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/644c1a84-c849-4a1f-8ba5-3c358d956678-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:14.676000 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.675957    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhcvq\" (UniqueName: \"kubernetes.io/projected/c194fbca-c1ac-4441-9ab4-1f902110b978-kube-api-access-hhcvq\") pod \"c194fbca-c1ac-4441-9ab4-1f902110b978\" (UID: \"c194fbca-c1ac-4441-9ab4-1f902110b978\") "
Sep 11 18:46:14.676167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.676028    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22d6s\" (UniqueName: \"kubernetes.io/projected/640bcbc4-85c5-4895-acbe-f2d5737193d4-kube-api-access-22d6s\") pod \"640bcbc4-85c5-4895-acbe-f2d5737193d4\" (UID: \"640bcbc4-85c5-4895-acbe-f2d5737193d4\") "
Sep 11 18:46:14.676167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.676054    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/640bcbc4-85c5-4895-acbe-f2d5737193d4-plugins-config-volume\") pod \"640bcbc4-85c5-4895-acbe-f2d5737193d4\" (UID: \"640bcbc4-85c5-4895-acbe-f2d5737193d4\") "
Sep 11 18:46:14.676167 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.676101    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c194fbca-c1ac-4441-9ab4-1f902110b978-plugins-config-volume\") pod \"c194fbca-c1ac-4441-9ab4-1f902110b978\" (UID: \"c194fbca-c1ac-4441-9ab4-1f902110b978\") "
Sep 11 18:46:14.676448 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.676417    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/640bcbc4-85c5-4895-acbe-f2d5737193d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "640bcbc4-85c5-4895-acbe-f2d5737193d4" (UID: "640bcbc4-85c5-4895-acbe-f2d5737193d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:46:14.676448 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.676441    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c194fbca-c1ac-4441-9ab4-1f902110b978-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c194fbca-c1ac-4441-9ab4-1f902110b978" (UID: "c194fbca-c1ac-4441-9ab4-1f902110b978"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:46:14.678115 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.678089    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c194fbca-c1ac-4441-9ab4-1f902110b978-kube-api-access-hhcvq" (OuterVolumeSpecName: "kube-api-access-hhcvq") pod "c194fbca-c1ac-4441-9ab4-1f902110b978" (UID: "c194fbca-c1ac-4441-9ab4-1f902110b978"). InnerVolumeSpecName "kube-api-access-hhcvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:46:14.678206 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.678130    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/640bcbc4-85c5-4895-acbe-f2d5737193d4-kube-api-access-22d6s" (OuterVolumeSpecName: "kube-api-access-22d6s") pod "640bcbc4-85c5-4895-acbe-f2d5737193d4" (UID: "640bcbc4-85c5-4895-acbe-f2d5737193d4"). InnerVolumeSpecName "kube-api-access-22d6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:46:14.777815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.777789    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhcvq\" (UniqueName: \"kubernetes.io/projected/c194fbca-c1ac-4441-9ab4-1f902110b978-kube-api-access-hhcvq\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:14.777815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.777812    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22d6s\" (UniqueName: \"kubernetes.io/projected/640bcbc4-85c5-4895-acbe-f2d5737193d4-kube-api-access-22d6s\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:14.777815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.777823    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/640bcbc4-85c5-4895-acbe-f2d5737193d4-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:14.778082 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:14.777831    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c194fbca-c1ac-4441-9ab4-1f902110b978-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:15.090935 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.090902    2578 generic.go:356] "Generic (PLEG): container finished" podID="640bcbc4-85c5-4895-acbe-f2d5737193d4" containerID="759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b" exitCode=0
Sep 11 18:46:15.091468 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.090946    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"
Sep 11 18:46:15.091468 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.090992    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb" event={"ID":"640bcbc4-85c5-4895-acbe-f2d5737193d4","Type":"ContainerDied","Data":"759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b"}
Sep 11 18:46:15.091468 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.091055    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb" event={"ID":"640bcbc4-85c5-4895-acbe-f2d5737193d4","Type":"ContainerDied","Data":"1fcfd6b75a2d272e5f2a5478c60a65d9d44e66e0991e9194ccbe1f2275387aaf"}
Sep 11 18:46:15.091468 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.091092    2578 scope.go:117] "RemoveContainer" containerID="759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b"
Sep 11 18:46:15.092819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.092608    2578 generic.go:356] "Generic (PLEG): container finished" podID="c194fbca-c1ac-4441-9ab4-1f902110b978" containerID="35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b" exitCode=0
Sep 11 18:46:15.092819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.092659    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"
Sep 11 18:46:15.092819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.092662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf" event={"ID":"c194fbca-c1ac-4441-9ab4-1f902110b978","Type":"ContainerDied","Data":"35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b"}
Sep 11 18:46:15.092819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.092727    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf" event={"ID":"c194fbca-c1ac-4441-9ab4-1f902110b978","Type":"ContainerDied","Data":"ccea27929357fbb4153a89562bd5a477d525f97b85a2f6b325a10e4fab1041d4"}
Sep 11 18:46:15.094894 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.094594    2578 generic.go:356] "Generic (PLEG): container finished" podID="644c1a84-c849-4a1f-8ba5-3c358d956678" containerID="03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa" exitCode=0
Sep 11 18:46:15.094894 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.094623    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql" event={"ID":"644c1a84-c849-4a1f-8ba5-3c358d956678","Type":"ContainerDied","Data":"03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa"}
Sep 11 18:46:15.094894 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.094643    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"
Sep 11 18:46:15.094894 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.094709    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql" event={"ID":"644c1a84-c849-4a1f-8ba5-3c358d956678","Type":"ContainerDied","Data":"93a714458b643b42770d139b6ce07d22496be0eaef716f55088305fcf29ec31f"}
Sep 11 18:46:15.111793 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.111774    2578 scope.go:117] "RemoveContainer" containerID="759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b"
Sep 11 18:46:15.112105 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:46:15.112085    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b\": container with ID starting with 759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b not found: ID does not exist" containerID="759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b"
Sep 11 18:46:15.112226 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.112116    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b"} err="failed to get container status \"759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b\": rpc error: code = NotFound desc = could not find container \"759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b\": container with ID starting with 759ed8e30c4c01670b23fd0148a80ded626976fb4bc9ef54231c43ac8d52019b not found: ID does not exist"
Sep 11 18:46:15.112226 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.112139    2578 scope.go:117] "RemoveContainer" containerID="35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b"
Sep 11 18:46:15.120269 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.120251    2578 scope.go:117] "RemoveContainer" containerID="35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b"
Sep 11 18:46:15.120532 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:46:15.120514    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b\": container with ID starting with 35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b not found: ID does not exist" containerID="35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b"
Sep 11 18:46:15.120581 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.120541    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b"} err="failed to get container status \"35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b\": rpc error: code = NotFound desc = could not find container \"35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b\": container with ID starting with 35920a789cdfe79350b3ae96ed214b87fa739e831fbd07b5cb9ae896a4a8e78b not found: ID does not exist"
Sep 11 18:46:15.120581 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.120558    2578 scope.go:117] "RemoveContainer" containerID="03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa"
Sep 11 18:46:15.127916 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.127897    2578 scope.go:117] "RemoveContainer" containerID="03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa"
Sep 11 18:46:15.128162 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:46:15.128144    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa\": container with ID starting with 03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa not found: ID does not exist" containerID="03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa"
Sep 11 18:46:15.128210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.128168    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa"} err="failed to get container status \"03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa\": rpc error: code = NotFound desc = could not find container \"03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa\": container with ID starting with 03b7c0b0e8758d18144b992aca1e6e484ffb6a826211b5fa225a3e8216d638aa not found: ID does not exist"
Sep 11 18:46:15.132476 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.132457    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"]
Sep 11 18:46:15.136999 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.136979    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72c98d-645974d6cf-5s7rf"]
Sep 11 18:46:15.145505 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.145486    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"]
Sep 11 18:46:15.149431 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.149411    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-72c98d-58968587d-pqplb"]
Sep 11 18:46:15.158873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.158853    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"]
Sep 11 18:46:15.162196 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.162175    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-200c07-6f7c7856cd-sdzql"]
Sep 11 18:46:15.409737 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.409711    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dec5a0b-c059-4e29-9d3d-5f1d28737c4d" path="/var/lib/kubelet/pods/4dec5a0b-c059-4e29-9d3d-5f1d28737c4d/volumes"
Sep 11 18:46:15.410048 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.410035    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="640bcbc4-85c5-4895-acbe-f2d5737193d4" path="/var/lib/kubelet/pods/640bcbc4-85c5-4895-acbe-f2d5737193d4/volumes"
Sep 11 18:46:15.410351 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.410340    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="644c1a84-c849-4a1f-8ba5-3c358d956678" path="/var/lib/kubelet/pods/644c1a84-c849-4a1f-8ba5-3c358d956678/volumes"
Sep 11 18:46:15.410627 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.410617    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="856d1224-5cc1-41f6-bd86-d875b5346eb5" path="/var/lib/kubelet/pods/856d1224-5cc1-41f6-bd86-d875b5346eb5/volumes"
Sep 11 18:46:15.410954 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.410942    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c194fbca-c1ac-4441-9ab4-1f902110b978" path="/var/lib/kubelet/pods/c194fbca-c1ac-4441-9ab4-1f902110b978/volumes"
Sep 11 18:46:15.411233 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:15.411222    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee5842dd-b491-4b78-a58a-c5132f8140e9" path="/var/lib/kubelet/pods/ee5842dd-b491-4b78-a58a-c5132f8140e9/volumes"
Sep 11 18:46:21.072598 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:21.072564    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"]
Sep 11 18:46:21.073108 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:21.072855    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy" containerID="cri-o://aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2" gracePeriod=30
Sep 11 18:46:22.002745 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.002712    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"]
Sep 11 18:46:22.003078 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003066    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy"
Sep 11 18:46:22.003078 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003079    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy"
Sep 11 18:46:22.003157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003093    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5842dd-b491-4b78-a58a-c5132f8140e9" containerName="revoke-keys"
Sep 11 18:46:22.003157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003098    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5842dd-b491-4b78-a58a-c5132f8140e9" containerName="revoke-keys"
Sep 11 18:46:22.003157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003104    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c194fbca-c1ac-4441-9ab4-1f902110b978" containerName="payload-processing"
Sep 11 18:46:22.003157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003111    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c194fbca-c1ac-4441-9ab4-1f902110b978" containerName="payload-processing"
Sep 11 18:46:22.003157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003123    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="644c1a84-c849-4a1f-8ba5-3c358d956678" containerName="payload-pre-processing"
Sep 11 18:46:22.003157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003129    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="644c1a84-c849-4a1f-8ba5-3c358d956678" containerName="payload-pre-processing"
Sep 11 18:46:22.003157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003137    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="640bcbc4-85c5-4895-acbe-f2d5737193d4" containerName="payload-pre-processing"
Sep 11 18:46:22.003157 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003142    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="640bcbc4-85c5-4895-acbe-f2d5737193d4" containerName="payload-pre-processing"
Sep 11 18:46:22.003399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003193    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy"
Sep 11 18:46:22.003399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003200    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee5842dd-b491-4b78-a58a-c5132f8140e9" containerName="revoke-keys"
Sep 11 18:46:22.003399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003206    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="644c1a84-c849-4a1f-8ba5-3c358d956678" containerName="payload-pre-processing"
Sep 11 18:46:22.003399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003212    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c194fbca-c1ac-4441-9ab4-1f902110b978" containerName="payload-processing"
Sep 11 18:46:22.003399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003220    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy"
Sep 11 18:46:22.003399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.003227    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="640bcbc4-85c5-4895-acbe-f2d5737193d4" containerName="payload-pre-processing"
Sep 11 18:46:22.006801 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.006783    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"
Sep 11 18:46:22.009312 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.009296    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:46:22.009403 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.009295    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:46:22.014228 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.014196    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"]
Sep 11 18:46:22.147543 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.147507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00e522ba-7f56-47c1-ba90-a0e749538912-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f\" (UID: \"00e522ba-7f56-47c1-ba90-a0e749538912\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"
Sep 11 18:46:22.248352 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.248324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00e522ba-7f56-47c1-ba90-a0e749538912-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f\" (UID: \"00e522ba-7f56-47c1-ba90-a0e749538912\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"
Sep 11 18:46:22.248982 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.248963    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00e522ba-7f56-47c1-ba90-a0e749538912-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f\" (UID: \"00e522ba-7f56-47c1-ba90-a0e749538912\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"
Sep 11 18:46:22.320334 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.320253    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"
Sep 11 18:46:22.610096 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:22.610066    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"]
Sep 11 18:46:22.611492 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:46:22.611467    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e522ba_7f56_47c1_ba90_a0e749538912.slice/crio-4c9e9f265e085b6ece99c522cff86358f24a8c6081764900759da4d04c6b4d9d WatchSource:0}: Error finding container 4c9e9f265e085b6ece99c522cff86358f24a8c6081764900759da4d04c6b4d9d: Status 404 returned error can't find the container with id 4c9e9f265e085b6ece99c522cff86358f24a8c6081764900759da4d04c6b4d9d
Sep 11 18:46:23.145368 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:23.145329    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f" event={"ID":"00e522ba-7f56-47c1-ba90-a0e749538912","Type":"ContainerStarted","Data":"b36824ce93f0da08346716554321a03550957afb497b4878816f5b024f3d34ec"}
Sep 11 18:46:23.145549 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:23.145419    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f" event={"ID":"00e522ba-7f56-47c1-ba90-a0e749538912","Type":"ContainerStarted","Data":"4c9e9f265e085b6ece99c522cff86358f24a8c6081764900759da4d04c6b4d9d"}
Sep 11 18:46:24.152747 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:24.152708    2578 generic.go:356] "Generic (PLEG): container finished" podID="00e522ba-7f56-47c1-ba90-a0e749538912" containerID="b36824ce93f0da08346716554321a03550957afb497b4878816f5b024f3d34ec" exitCode=0
Sep 11 18:46:24.153186 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:24.152838    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f" event={"ID":"00e522ba-7f56-47c1-ba90-a0e749538912","Type":"ContainerDied","Data":"b36824ce93f0da08346716554321a03550957afb497b4878816f5b024f3d34ec"}
Sep 11 18:46:25.303950 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:25.303927    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"
Sep 11 18:46:25.481974 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:25.481893    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00e522ba-7f56-47c1-ba90-a0e749538912-maas-api-service-ca\") pod \"00e522ba-7f56-47c1-ba90-a0e749538912\" (UID: \"00e522ba-7f56-47c1-ba90-a0e749538912\") "
Sep 11 18:46:25.482229 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:25.482207    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00e522ba-7f56-47c1-ba90-a0e749538912-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "00e522ba-7f56-47c1-ba90-a0e749538912" (UID: "00e522ba-7f56-47c1-ba90-a0e749538912"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:46:25.582761 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:25.582724    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00e522ba-7f56-47c1-ba90-a0e749538912-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:26.171287 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.171260    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"
Sep 11 18:46:26.171454 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.171261    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f" event={"ID":"00e522ba-7f56-47c1-ba90-a0e749538912","Type":"ContainerDied","Data":"4c9e9f265e085b6ece99c522cff86358f24a8c6081764900759da4d04c6b4d9d"}
Sep 11 18:46:26.171454 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.171390    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c9e9f265e085b6ece99c522cff86358f24a8c6081764900759da4d04c6b4d9d"
Sep 11 18:46:26.321483 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.321461    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:46:26.491166 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491134    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istiod-ca-cert\") pod \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") "
Sep 11 18:46:26.491343 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491177    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-socket\") pod \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") "
Sep 11 18:46:26.491343 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491215    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-podinfo\") pod \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") "
Sep 11 18:46:26.491343 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491246    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfx7d\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-kube-api-access-vfx7d\") pod \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") "
Sep 11 18:46:26.491343 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491278    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-data\") pod \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") "
Sep 11 18:46:26.491343 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491296    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-credential-socket\") pod \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") "
Sep 11 18:46:26.491343 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491311    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-envoy\") pod \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") "
Sep 11 18:46:26.491600 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491363    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-token\") pod \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") "
Sep 11 18:46:26.491600 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491395    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-certs\") pod \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\" (UID: \"44d90e55-a5ce-4d17-9c2f-8b46344193c5\") "
Sep 11 18:46:26.491600 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491585    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44d90e55-a5ce-4d17-9c2f-8b46344193c5" (UID: "44d90e55-a5ce-4d17-9c2f-8b46344193c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:46:26.491789 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491590    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44d90e55-a5ce-4d17-9c2f-8b46344193c5" (UID: "44d90e55-a5ce-4d17-9c2f-8b46344193c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:26.491789 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491663    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-data" (OuterVolumeSpecName: "istio-data") pod "44d90e55-a5ce-4d17-9c2f-8b46344193c5" (UID: "44d90e55-a5ce-4d17-9c2f-8b46344193c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:26.491789 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491694    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44d90e55-a5ce-4d17-9c2f-8b46344193c5" (UID: "44d90e55-a5ce-4d17-9c2f-8b46344193c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:26.491789 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491716    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:26.491789 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491747    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:26.492028 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.491943    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44d90e55-a5ce-4d17-9c2f-8b46344193c5" (UID: "44d90e55-a5ce-4d17-9c2f-8b46344193c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:26.493603 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.493574    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44d90e55-a5ce-4d17-9c2f-8b46344193c5" (UID: "44d90e55-a5ce-4d17-9c2f-8b46344193c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:46:26.493777 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.493745    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "44d90e55-a5ce-4d17-9c2f-8b46344193c5" (UID: "44d90e55-a5ce-4d17-9c2f-8b46344193c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:46:26.493874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.493796    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-kube-api-access-vfx7d" (OuterVolumeSpecName: "kube-api-access-vfx7d") pod "44d90e55-a5ce-4d17-9c2f-8b46344193c5" (UID: "44d90e55-a5ce-4d17-9c2f-8b46344193c5"). InnerVolumeSpecName "kube-api-access-vfx7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:46:26.493874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.493809    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44d90e55-a5ce-4d17-9c2f-8b46344193c5" (UID: "44d90e55-a5ce-4d17-9c2f-8b46344193c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:46:26.592663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.592632    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:26.592663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.592659    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:26.592663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.592668    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:26.592896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.592690    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:26.592896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.592702    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d90e55-a5ce-4d17-9c2f-8b46344193c5-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:26.592896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.592713    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d90e55-a5ce-4d17-9c2f-8b46344193c5-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:26.592896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:26.592725    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfx7d\" (UniqueName: \"kubernetes.io/projected/44d90e55-a5ce-4d17-9c2f-8b46344193c5-kube-api-access-vfx7d\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:27.178652 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.178615    2578 generic.go:356] "Generic (PLEG): container finished" podID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerID="aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2" exitCode=0
Sep 11 18:46:27.178841 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.178661    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"
Sep 11 18:46:27.178841 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.178660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" event={"ID":"44d90e55-a5ce-4d17-9c2f-8b46344193c5","Type":"ContainerDied","Data":"aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2"}
Sep 11 18:46:27.178841 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.178802    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl" event={"ID":"44d90e55-a5ce-4d17-9c2f-8b46344193c5","Type":"ContainerDied","Data":"085c5ad5dd17c907bd15bee582c14d29e369329f1851f621be72e0eb721922c6"}
Sep 11 18:46:27.178841 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.178834    2578 scope.go:117] "RemoveContainer" containerID="aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2"
Sep 11 18:46:27.188781 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.188603    2578 scope.go:117] "RemoveContainer" containerID="45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1"
Sep 11 18:46:27.197223 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.197203    2578 scope.go:117] "RemoveContainer" containerID="aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2"
Sep 11 18:46:27.197500 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:46:27.197482    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2\": container with ID starting with aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2 not found: ID does not exist" containerID="aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2"
Sep 11 18:46:27.197552 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.197507    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2"} err="failed to get container status \"aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2\": rpc error: code = NotFound desc = could not find container \"aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2\": container with ID starting with aa78ecb93f4e86b1bd05639cc2f16a82e4f640349d6c277bebe4420c27d769f2 not found: ID does not exist"
Sep 11 18:46:27.197552 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.197523    2578 scope.go:117] "RemoveContainer" containerID="45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1"
Sep 11 18:46:27.197753 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:46:27.197733    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1\": container with ID starting with 45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1 not found: ID does not exist" containerID="45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1"
Sep 11 18:46:27.197828 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.197764    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1"} err="failed to get container status \"45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1\": rpc error: code = NotFound desc = could not find container \"45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1\": container with ID starting with 45d6defa444834deb05e1df87c9fd6b89739b32daa0a47db382385c587ce05f1 not found: ID does not exist"
Sep 11 18:46:27.201209 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.201187    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"]
Sep 11 18:46:27.205123 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.205102    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-200c07-openshift-default-7b76f5dfb8-vd6nl"]
Sep 11 18:46:27.411326 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:27.411296    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" path="/var/lib/kubelet/pods/44d90e55-a5ce-4d17-9c2f-8b46344193c5/volumes"
Sep 11 18:46:32.649388 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.649350    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx"]
Sep 11 18:46:32.652987 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.652962    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-238bae-29819205-jhnsx"]
Sep 11 18:46:32.678342 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.678309    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"]
Sep 11 18:46:32.678630 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.678570    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786" podUID="2e21afd3-5122-4cf0-9f22-1a2ae1b03336" containerName="payload-processing" containerID="cri-o://3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d" gracePeriod=30
Sep 11 18:46:32.852476 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.852443    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"]
Sep 11 18:46:32.856529 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.856503    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-238bae-9qv4f"]
Sep 11 18:46:32.931221 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.931197    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:46:32.951628 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.951608    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-plugins-config-volume\") pod \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\" (UID: \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\") "
Sep 11 18:46:32.951752 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.951660    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmsbq\" (UniqueName: \"kubernetes.io/projected/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-kube-api-access-zmsbq\") pod \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\" (UID: \"2e21afd3-5122-4cf0-9f22-1a2ae1b03336\") "
Sep 11 18:46:32.952018 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.951992    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e21afd3-5122-4cf0-9f22-1a2ae1b03336" (UID: "2e21afd3-5122-4cf0-9f22-1a2ae1b03336"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:46:32.953749 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:32.953722    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-kube-api-access-zmsbq" (OuterVolumeSpecName: "kube-api-access-zmsbq") pod "2e21afd3-5122-4cf0-9f22-1a2ae1b03336" (UID: "2e21afd3-5122-4cf0-9f22-1a2ae1b03336"). InnerVolumeSpecName "kube-api-access-zmsbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:46:33.052540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.052505    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:33.052540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.052531    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmsbq\" (UniqueName: \"kubernetes.io/projected/2e21afd3-5122-4cf0-9f22-1a2ae1b03336-kube-api-access-zmsbq\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:46:33.214515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.214432    2578 generic.go:356] "Generic (PLEG): container finished" podID="2e21afd3-5122-4cf0-9f22-1a2ae1b03336" containerID="3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d" exitCode=0
Sep 11 18:46:33.214515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.214479    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"
Sep 11 18:46:33.214515 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.214495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786" event={"ID":"2e21afd3-5122-4cf0-9f22-1a2ae1b03336","Type":"ContainerDied","Data":"3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d"}
Sep 11 18:46:33.214750 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.214537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786" event={"ID":"2e21afd3-5122-4cf0-9f22-1a2ae1b03336","Type":"ContainerDied","Data":"2e14317a339e46c8a66a1d5a475b4be40c0f63bc91b0c6158f0b992b07f0cded"}
Sep 11 18:46:33.214750 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.214558    2578 scope.go:117] "RemoveContainer" containerID="3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d"
Sep 11 18:46:33.229855 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.229837    2578 scope.go:117] "RemoveContainer" containerID="3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d"
Sep 11 18:46:33.230146 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:46:33.230126    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d\": container with ID starting with 3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d not found: ID does not exist" containerID="3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d"
Sep 11 18:46:33.230206 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.230157    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d"} err="failed to get container status \"3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d\": rpc error: code = NotFound desc = could not find container \"3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d\": container with ID starting with 3a85c28fc174eabd1eb692b2c13475865ef77b00679e31b4a3b649d04836344d not found: ID does not exist"
Sep 11 18:46:33.238491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.238469    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"]
Sep 11 18:46:33.242627 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.242599    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-238bae-58db4b64df-tj786"]
Sep 11 18:46:33.409801 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.409771    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00e522ba-7f56-47c1-ba90-a0e749538912" path="/var/lib/kubelet/pods/00e522ba-7f56-47c1-ba90-a0e749538912/volumes"
Sep 11 18:46:33.410090 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.410076    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19d9e4fc-4fd6-4afa-a89b-0f4a37d8ac02" path="/var/lib/kubelet/pods/19d9e4fc-4fd6-4afa-a89b-0f4a37d8ac02/volumes"
Sep 11 18:46:33.410372 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:33.410361    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e21afd3-5122-4cf0-9f22-1a2ae1b03336" path="/var/lib/kubelet/pods/2e21afd3-5122-4cf0-9f22-1a2ae1b03336/volumes"
Sep 11 18:46:37.768252 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:37.768220    2578 scope.go:117] "RemoveContainer" containerID="63ff34ce224d2bf57b578e516f91bf1b13074c5e5f9f8b3131e96282f868c6b5"
Sep 11 18:46:48.952113 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:48.952071    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"]
Sep 11 18:46:49.837342 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:46:49.837308    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"]
Sep 11 18:47:03.971746 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:03.971606    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq" podUID="d03be4dc-8b14-42d9-af84-673967b7af14" containerName="main" containerID="cri-o://7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9" gracePeriod=45
Sep 11 18:47:04.219019 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.218991    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:47:04.352280 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.352187    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-dshm\") pod \"d03be4dc-8b14-42d9-af84-673967b7af14\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") "
Sep 11 18:47:04.352280 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.352232    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5s6g\" (UniqueName: \"kubernetes.io/projected/d03be4dc-8b14-42d9-af84-673967b7af14-kube-api-access-h5s6g\") pod \"d03be4dc-8b14-42d9-af84-673967b7af14\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") "
Sep 11 18:47:04.352280 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.352278    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-model-cache\") pod \"d03be4dc-8b14-42d9-af84-673967b7af14\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") "
Sep 11 18:47:04.352644 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.352317    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d03be4dc-8b14-42d9-af84-673967b7af14-tls-certs\") pod \"d03be4dc-8b14-42d9-af84-673967b7af14\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") "
Sep 11 18:47:04.352644 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.352389    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-tmp-dir\") pod \"d03be4dc-8b14-42d9-af84-673967b7af14\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") "
Sep 11 18:47:04.352644 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.352443    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-home\") pod \"d03be4dc-8b14-42d9-af84-673967b7af14\" (UID: \"d03be4dc-8b14-42d9-af84-673967b7af14\") "
Sep 11 18:47:04.352644 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.352574    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-model-cache" (OuterVolumeSpecName: "model-cache") pod "d03be4dc-8b14-42d9-af84-673967b7af14" (UID: "d03be4dc-8b14-42d9-af84-673967b7af14"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:47:04.353364 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.353318    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d03be4dc-8b14-42d9-af84-673967b7af14" (UID: "d03be4dc-8b14-42d9-af84-673967b7af14"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:47:04.353495 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.353412    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-home" (OuterVolumeSpecName: "home") pod "d03be4dc-8b14-42d9-af84-673967b7af14" (UID: "d03be4dc-8b14-42d9-af84-673967b7af14"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:47:04.353663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.353637    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-model-cache\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:04.356380 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.354967    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-dshm" (OuterVolumeSpecName: "dshm") pod "d03be4dc-8b14-42d9-af84-673967b7af14" (UID: "d03be4dc-8b14-42d9-af84-673967b7af14"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:47:04.362661 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.362585    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d03be4dc-8b14-42d9-af84-673967b7af14-kube-api-access-h5s6g" (OuterVolumeSpecName: "kube-api-access-h5s6g") pod "d03be4dc-8b14-42d9-af84-673967b7af14" (UID: "d03be4dc-8b14-42d9-af84-673967b7af14"). InnerVolumeSpecName "kube-api-access-h5s6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:47:04.363299 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.363267    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d03be4dc-8b14-42d9-af84-673967b7af14-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d03be4dc-8b14-42d9-af84-673967b7af14" (UID: "d03be4dc-8b14-42d9-af84-673967b7af14"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:47:04.386115 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.386083    2578 generic.go:356] "Generic (PLEG): container finished" podID="d03be4dc-8b14-42d9-af84-673967b7af14" containerID="7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9" exitCode=0
Sep 11 18:47:04.386287 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.386135    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"
Sep 11 18:47:04.386287 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.386167    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq" event={"ID":"d03be4dc-8b14-42d9-af84-673967b7af14","Type":"ContainerDied","Data":"7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9"}
Sep 11 18:47:04.386287 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.386210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq" event={"ID":"d03be4dc-8b14-42d9-af84-673967b7af14","Type":"ContainerDied","Data":"a9f70679519254029d16cb436efa2716faaeef44b83033dffb82f1b43f6d4c47"}
Sep 11 18:47:04.386287 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.386228    2578 scope.go:117] "RemoveContainer" containerID="7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9"
Sep 11 18:47:04.395159 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.395143    2578 scope.go:117] "RemoveContainer" containerID="7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9"
Sep 11 18:47:04.395421 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:47:04.395404    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9\": container with ID starting with 7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9 not found: ID does not exist" containerID="7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9"
Sep 11 18:47:04.395471 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.395428    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9"} err="failed to get container status \"7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9\": rpc error: code = NotFound desc = could not find container \"7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9\": container with ID starting with 7ff029bcb66a7481ff6a502af8b5533aac63c16f46afabc6d27d2695b77fd1a9 not found: ID does not exist"
Sep 11 18:47:04.409870 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.409840    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"]
Sep 11 18:47:04.413081 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.413059    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d90e71/auth-test-model-d90e71-kserve-84cb978c-nh7wq"]
Sep 11 18:47:04.455012 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.454979    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d03be4dc-8b14-42d9-af84-673967b7af14-tls-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:04.455151 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.455028    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-tmp-dir\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:04.455151 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.455037    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-home\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:04.455151 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.455045    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d03be4dc-8b14-42d9-af84-673967b7af14-dshm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:04.455151 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.455054    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5s6g\" (UniqueName: \"kubernetes.io/projected/d03be4dc-8b14-42d9-af84-673967b7af14-kube-api-access-h5s6g\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:04.856903 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:04.856861    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6" podUID="4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" containerName="main" containerID="cri-o://80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9" gracePeriod=45
Sep 11 18:47:05.116615 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.116538    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:47:05.159160 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.159122    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-home\") pod \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") "
Sep 11 18:47:05.159363 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.159188    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-dshm\") pod \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") "
Sep 11 18:47:05.159363 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.159231    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-model-cache\") pod \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") "
Sep 11 18:47:05.159363 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.159272    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tls-certs\") pod \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") "
Sep 11 18:47:05.159363 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.159302    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tmp-dir\") pod \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") "
Sep 11 18:47:05.159363 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.159334    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr2zj\" (UniqueName: \"kubernetes.io/projected/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-kube-api-access-wr2zj\") pod \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\" (UID: \"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0\") "
Sep 11 18:47:05.159616 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.159561    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" (UID: "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:47:05.159671 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.159610    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" (UID: "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:47:05.159671 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.159433    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-home" (OuterVolumeSpecName: "home") pod "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" (UID: "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:47:05.161903 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.161866    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-kube-api-access-wr2zj" (OuterVolumeSpecName: "kube-api-access-wr2zj") pod "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" (UID: "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0"). InnerVolumeSpecName "kube-api-access-wr2zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:47:05.162022 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.161931    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-dshm" (OuterVolumeSpecName: "dshm") pod "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" (UID: "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:47:05.162022 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.161942    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" (UID: "4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:47:05.260355 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.260298    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tls-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:05.260355 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.260347    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-tmp-dir\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:05.260355 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.260357    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr2zj\" (UniqueName: \"kubernetes.io/projected/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-kube-api-access-wr2zj\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:05.260355 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.260366    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-home\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:05.260355 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.260377    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-dshm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:05.260709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.260384    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0-model-cache\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:47:05.394496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.394399    2578 generic.go:356] "Generic (PLEG): container finished" podID="4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" containerID="80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9" exitCode=0
Sep 11 18:47:05.394496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.394443    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"
Sep 11 18:47:05.394496 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.394459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6" event={"ID":"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0","Type":"ContainerDied","Data":"80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9"}
Sep 11 18:47:05.394793 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.394499    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6" event={"ID":"4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0","Type":"ContainerDied","Data":"3c2174498ae5f539abf5042b4adc5312fe4b2cc88197190fbba1f5e1033bc9c5"}
Sep 11 18:47:05.394793 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.394522    2578 scope.go:117] "RemoveContainer" containerID="80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9"
Sep 11 18:47:05.403998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.403972    2578 scope.go:117] "RemoveContainer" containerID="80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9"
Sep 11 18:47:05.404329 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:47:05.404304    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9\": container with ID starting with 80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9 not found: ID does not exist" containerID="80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9"
Sep 11 18:47:05.404399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.404342    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9"} err="failed to get container status \"80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9\": rpc error: code = NotFound desc = could not find container \"80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9\": container with ID starting with 80a5ee823212219ab6e7cc0add29026225f25fcde4227c058f63afcdb9c87ea9 not found: ID does not exist"
Sep 11 18:47:05.409755 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.409731    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d03be4dc-8b14-42d9-af84-673967b7af14" path="/var/lib/kubelet/pods/d03be4dc-8b14-42d9-af84-673967b7af14/volumes"
Sep 11 18:47:05.416705 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.416661    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"]
Sep 11 18:47:05.421275 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:05.421250    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/auth-test-model-553671-kserve-6c5b8b9f79-sspn6"]
Sep 11 18:47:07.411130 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:07.411095    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" path="/var/lib/kubelet/pods/4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0/volumes"
Sep 11 18:47:37.820218 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:37.820186    2578 scope.go:117] "RemoveContainer" containerID="b7ee72b4ac32e558a966eff0accfc0654e5deb52e461a53b5843db01baf54aa6"
Sep 11 18:47:41.347464 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347426    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"]
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347796    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d03be4dc-8b14-42d9-af84-673967b7af14" containerName="main"
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347808    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d03be4dc-8b14-42d9-af84-673967b7af14" containerName="main"
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347820    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" containerName="main"
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347826    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" containerName="main"
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347838    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e21afd3-5122-4cf0-9f22-1a2ae1b03336" containerName="payload-processing"
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347844    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e21afd3-5122-4cf0-9f22-1a2ae1b03336" containerName="payload-processing"
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347853    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy"
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347858    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8030c458-6d3d-44ed-97ca-a92701958f74" containerName="istio-proxy"
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347868    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00e522ba-7f56-47c1-ba90-a0e749538912" containerName="revoke-keys"
Sep 11 18:47:41.347874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347874    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="00e522ba-7f56-47c1-ba90-a0e749538912" containerName="revoke-keys"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347881    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347886    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347895    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347900    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347956    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f940308-7457-4ef9-8d5e-2b4c7cdbfbb0" containerName="main"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347971    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347978    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e21afd3-5122-4cf0-9f22-1a2ae1b03336" containerName="payload-processing"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347983    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d90e55-a5ce-4d17-9c2f-8b46344193c5" containerName="istio-proxy"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347989    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d03be4dc-8b14-42d9-af84-673967b7af14" containerName="main"
Sep 11 18:47:41.348239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.347994    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="00e522ba-7f56-47c1-ba90-a0e749538912" containerName="revoke-keys"
Sep 11 18:47:41.350941 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.350926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.353381 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.353359    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"openshift-service-ca.crt\""
Sep 11 18:47:41.353506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.353487    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"kube-root-ca.crt\""
Sep 11 18:47:41.354207 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.354188    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"sub-test-model-553671-kserve-self-signed-certs\""
Sep 11 18:47:41.354207 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.354198    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"default-dockercfg-mfbjv\""
Sep 11 18:47:41.361612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.361585    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"]
Sep 11 18:47:41.489216 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.489181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7vld\" (UniqueName: \"kubernetes.io/projected/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-kube-api-access-f7vld\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.489216 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.489219    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-home\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.489480 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.489241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-model-cache\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.489480 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.489344    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-dshm\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.489480 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.489383    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tls-certs\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.489480 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.489413    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tmp-dir\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.589994 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.589951    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-dshm\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.589994 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.589992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tls-certs\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.590266 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.590226    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tmp-dir\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.590395 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.590378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7vld\" (UniqueName: \"kubernetes.io/projected/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-kube-api-access-f7vld\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.590466 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.590408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-home\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.590466 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.590428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-model-cache\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.590594 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.590564    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tmp-dir\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.590784 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.590712    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-home\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.590854 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.590832    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-model-cache\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.592312 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.592289    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-dshm\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.592578 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.592560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tls-certs\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.598894 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.598848    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7vld\" (UniqueName: \"kubernetes.io/projected/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-kube-api-access-f7vld\") pod \"sub-test-model-553671-kserve-7d4bcb48bf-5kr42\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") " pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.660489 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.660452    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:41.932836 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:41.932653    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"]
Sep 11 18:47:41.935578 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:47:41.935536    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod432f2c74_2e0e_4e70_8cf5_29a584c8f7dd.slice/crio-a629d2cddc21a32cd939f27465a97b7faa564cc6b597d32e1d1d20b8b0114cd6 WatchSource:0}: Error finding container a629d2cddc21a32cd939f27465a97b7faa564cc6b597d32e1d1d20b8b0114cd6: Status 404 returned error can't find the container with id a629d2cddc21a32cd939f27465a97b7faa564cc6b597d32e1d1d20b8b0114cd6
Sep 11 18:47:42.597120 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:42.597084    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42" event={"ID":"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd","Type":"ContainerStarted","Data":"4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d"}
Sep 11 18:47:42.597501 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:42.597136    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42" event={"ID":"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd","Type":"ContainerStarted","Data":"a629d2cddc21a32cd939f27465a97b7faa564cc6b597d32e1d1d20b8b0114cd6"}
Sep 11 18:47:42.616856 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:42.616807    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42" podStartSLOduration=1.616791479 podStartE2EDuration="1.616791479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:47:42.614420024 +0000 UTC m=+1925.649636693" watchObservedRunningTime="2026-09-11 18:47:42.616791479 +0000 UTC m=+1925.652008200"
Sep 11 18:47:51.660576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:51.660538    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:51.661308 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:51.660595    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:51.673187 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:51.673165    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:47:52.659822 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:47:52.659796    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:48:26.939696 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:26.934415    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"]
Sep 11 18:48:37.847419 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:37.847338    2578 scope.go:117] "RemoveContainer" containerID="72d910f60a65240152b5522b1c4bc30bd325a18bd9509fe63f6067511cc50b9a"
Sep 11 18:48:41.960227 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:41.960178    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42" podUID="432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" containerName="main" containerID="cri-o://4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d" gracePeriod=45
Sep 11 18:48:42.203991 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.203963    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:48:42.247387 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.247305    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-home\") pod \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") "
Sep 11 18:48:42.247387 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.247376    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-model-cache\") pod \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") "
Sep 11 18:48:42.247612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.247408    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-dshm\") pod \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") "
Sep 11 18:48:42.247612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.247425    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tmp-dir\") pod \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") "
Sep 11 18:48:42.247612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.247475    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7vld\" (UniqueName: \"kubernetes.io/projected/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-kube-api-access-f7vld\") pod \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") "
Sep 11 18:48:42.247612 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.247497    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tls-certs\") pod \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\" (UID: \"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd\") "
Sep 11 18:48:42.247858 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.247662    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-home" (OuterVolumeSpecName: "home") pod "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" (UID: "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:48:42.247858 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.247719    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" (UID: "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:48:42.247858 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.247792    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" (UID: "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:48:42.249608 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.249584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-dshm" (OuterVolumeSpecName: "dshm") pod "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" (UID: "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:48:42.249767 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.249655    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-kube-api-access-f7vld" (OuterVolumeSpecName: "kube-api-access-f7vld") pod "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" (UID: "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd"). InnerVolumeSpecName "kube-api-access-f7vld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:48:42.249767 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.249739    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" (UID: "432f2c74-2e0e-4e70-8cf5-29a584c8f7dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:48:42.348637 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.348604    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-model-cache\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:48:42.348637 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.348632    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-dshm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:48:42.348637 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.348641    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tmp-dir\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:48:42.348909 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.348649    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7vld\" (UniqueName: \"kubernetes.io/projected/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-kube-api-access-f7vld\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:48:42.348909 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.348658    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-tls-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:48:42.348909 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.348667    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd-home\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:48:42.923120 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.923091    2578 generic.go:356] "Generic (PLEG): container finished" podID="432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" containerID="4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d" exitCode=0
Sep 11 18:48:42.923320 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.923132    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"
Sep 11 18:48:42.923320 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.923178    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42" event={"ID":"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd","Type":"ContainerDied","Data":"4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d"}
Sep 11 18:48:42.923320 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.923221    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42" event={"ID":"432f2c74-2e0e-4e70-8cf5-29a584c8f7dd","Type":"ContainerDied","Data":"a629d2cddc21a32cd939f27465a97b7faa564cc6b597d32e1d1d20b8b0114cd6"}
Sep 11 18:48:42.923320 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.923242    2578 scope.go:117] "RemoveContainer" containerID="4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d"
Sep 11 18:48:42.933027 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.933009    2578 scope.go:117] "RemoveContainer" containerID="4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d"
Sep 11 18:48:42.933315 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:48:42.933295    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d\": container with ID starting with 4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d not found: ID does not exist" containerID="4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d"
Sep 11 18:48:42.933355 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.933326    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d"} err="failed to get container status \"4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d\": rpc error: code = NotFound desc = could not find container \"4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d\": container with ID starting with 4aaf2e857c242687ddc4672301b896fda1473c8a1aab31e52c61a4fd3dc2ba8d not found: ID does not exist"
Sep 11 18:48:42.943590 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.943569    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"]
Sep 11 18:48:42.947299 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:42.947280    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/sub-test-model-553671-kserve-7d4bcb48bf-5kr42"]
Sep 11 18:48:43.410723 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:48:43.410697    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" path="/var/lib/kubelet/pods/432f2c74-2e0e-4e70-8cf5-29a584c8f7dd/volumes"
Sep 11 18:49:20.147574 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.147536    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"]
Sep 11 18:49:20.148171 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.147913    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" containerName="main"
Sep 11 18:49:20.148171 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.147926    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" containerName="main"
Sep 11 18:49:20.148171 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.148002    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="432f2c74-2e0e-4e70-8cf5-29a584c8f7dd" containerName="main"
Sep 11 18:49:20.151056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.151034    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.154243 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.154220    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"default-dockercfg-mfbjv\""
Sep 11 18:49:20.154243 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.154252    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"openshift-service-ca.crt\""
Sep 11 18:49:20.154431 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.154269    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"rate-test-model-553671-kserve-self-signed-certs\""
Sep 11 18:49:20.154431 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.154270    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-553671\"/\"kube-root-ca.crt\""
Sep 11 18:49:20.159117 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.159095    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"]
Sep 11 18:49:20.298114 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.298077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9674211-9bc0-4c80-a680-d2a3aef41eed-tls-certs\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.298317 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.298137    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-dshm\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.298317 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.298224    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh6rw\" (UniqueName: \"kubernetes.io/projected/a9674211-9bc0-4c80-a680-d2a3aef41eed-kube-api-access-xh6rw\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.298317 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.298274    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-home\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.298317 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.298308    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-model-cache\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.298520 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.298400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-tmp-dir\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.399938 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.399843    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-dshm\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.399938 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.399889    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh6rw\" (UniqueName: \"kubernetes.io/projected/a9674211-9bc0-4c80-a680-d2a3aef41eed-kube-api-access-xh6rw\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.399938 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.399908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-home\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.400240 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.400019    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-model-cache\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.400240 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.400106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-tmp-dir\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.400240 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.400153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9674211-9bc0-4c80-a680-d2a3aef41eed-tls-certs\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.400374 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.400255    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-home\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.400415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.400373    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-model-cache\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.400415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.400407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-tmp-dir\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.402127 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.402105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-dshm\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.402469 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.402450    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9674211-9bc0-4c80-a680-d2a3aef41eed-tls-certs\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.406983 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.406964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh6rw\" (UniqueName: \"kubernetes.io/projected/a9674211-9bc0-4c80-a680-d2a3aef41eed-kube-api-access-xh6rw\") pod \"rate-test-model-553671-kserve-664bc6b7c5-vvfbs\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") " pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.461625 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.461579    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:20.743991 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:20.743957    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"]
Sep 11 18:49:20.746030 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:49:20.746000    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9674211_9bc0_4c80_a680_d2a3aef41eed.slice/crio-f2ecccff0d3f71e7e69e803bb1e796d56bd91a96e7cbee70ccb73bd35b12daf5 WatchSource:0}: Error finding container f2ecccff0d3f71e7e69e803bb1e796d56bd91a96e7cbee70ccb73bd35b12daf5: Status 404 returned error can't find the container with id f2ecccff0d3f71e7e69e803bb1e796d56bd91a96e7cbee70ccb73bd35b12daf5
Sep 11 18:49:21.134080 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:21.133986    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs" event={"ID":"a9674211-9bc0-4c80-a680-d2a3aef41eed","Type":"ContainerStarted","Data":"7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd"}
Sep 11 18:49:21.134080 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:21.134038    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs" event={"ID":"a9674211-9bc0-4c80-a680-d2a3aef41eed","Type":"ContainerStarted","Data":"f2ecccff0d3f71e7e69e803bb1e796d56bd91a96e7cbee70ccb73bd35b12daf5"}
Sep 11 18:49:21.151497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:21.151442    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs" podStartSLOduration=1.15142133 podStartE2EDuration="1.15142133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:49:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:49:21.150858372 +0000 UTC m=+2024.186075043" watchObservedRunningTime="2026-09-11 18:49:21.15142133 +0000 UTC m=+2024.186638001"
Sep 11 18:49:30.462367 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:30.462325    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:30.462806 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:30.462379    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:30.474877 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:30.474851    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:49:31.200074 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:49:31.200043    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:50:06.526234 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:06.526155    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"]
Sep 11 18:50:07.273369 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.273334    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"]
Sep 11 18:50:07.277206 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.277179    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.280154 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.280128    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-da4d13-openshift-default-dockercfg-prvw6\""
Sep 11 18:50:07.295100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.295074    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"]
Sep 11 18:50:07.435023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.434985    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-credential-socket\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.435023 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.435023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-podinfo\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.435276 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.435044    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-socket\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.435276 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.435135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-token\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.435276 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.435163    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-certs\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.435276 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.435276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49c860a-1dde-40bc-851d-9c5697f9684f-istiod-ca-cert\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.435506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.435340    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-data\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.435506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.435394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-envoy\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.435506 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.435420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t42v\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-kube-api-access-7t42v\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.536552 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-data\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.536552 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-envoy\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.536552 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t42v\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-kube-api-access-7t42v\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536580    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-credential-socket\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-podinfo\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-socket\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-token\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-certs\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49c860a-1dde-40bc-851d-9c5697f9684f-istiod-ca-cert\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.536936    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-data\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.537020    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-credential-socket\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537148 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.537059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-socket\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537488 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.537263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-certs\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.537614 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.537592    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49c860a-1dde-40bc-851d-9c5697f9684f-istiod-ca-cert\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.539107 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.539085    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-envoy\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.539168 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.539095    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-podinfo\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.546260 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.546235    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t42v\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-kube-api-access-7t42v\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.546595 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.546573    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-token\") pod \"e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") " pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.587756 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.587734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:07.863101 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.863063    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"]
Sep 11 18:50:07.865598 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:50:07.865562    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49c860a_1dde_40bc_851d_9c5697f9684f.slice/crio-131f16a282c9e8f74cf080a5554b615076aa7278f3b418c66def66c4c0b74a38 WatchSource:0}: Error finding container 131f16a282c9e8f74cf080a5554b615076aa7278f3b418c66def66c4c0b74a38: Status 404 returned error can't find the container with id 131f16a282c9e8f74cf080a5554b615076aa7278f3b418c66def66c4c0b74a38
Sep 11 18:50:07.868125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.868077    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:07.868239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.868160    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:07.868239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:07.868192    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:08.392796 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:08.392755    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" event={"ID":"e49c860a-1dde-40bc-851d-9c5697f9684f","Type":"ContainerStarted","Data":"3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6"}
Sep 11 18:50:08.393010 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:08.392816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" event={"ID":"e49c860a-1dde-40bc-851d-9c5697f9684f","Type":"ContainerStarted","Data":"131f16a282c9e8f74cf080a5554b615076aa7278f3b418c66def66c4c0b74a38"}
Sep 11 18:50:08.425124 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:08.425071    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" podStartSLOduration=1.425052559 podStartE2EDuration="1.425052559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:50:08.423061541 +0000 UTC m=+2071.458278212" watchObservedRunningTime="2026-09-11 18:50:08.425052559 +0000 UTC m=+2071.460269293"
Sep 11 18:50:08.588826 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:08.588790    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:08.593664 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:08.593640    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:09.400390 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:09.400356    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:09.401240 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:09.401221    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:13.380801 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.380768    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"]
Sep 11 18:50:13.384239 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.384221    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:13.386933 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.386903    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-da4d13\""
Sep 11 18:50:13.387218 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.387200    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-da4d13-dockercfg-xskms\""
Sep 11 18:50:13.401415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.401386    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"]
Sep 11 18:50:13.494395 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.494350    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b60be98-7d0a-4e77-af20-5cf501487c84-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd\" (UID: \"2b60be98-7d0a-4e77-af20-5cf501487c84\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:13.494597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.494500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-249mq\" (UniqueName: \"kubernetes.io/projected/2b60be98-7d0a-4e77-af20-5cf501487c84-kube-api-access-249mq\") pod \"payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd\" (UID: \"2b60be98-7d0a-4e77-af20-5cf501487c84\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:13.595385 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.595348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249mq\" (UniqueName: \"kubernetes.io/projected/2b60be98-7d0a-4e77-af20-5cf501487c84-kube-api-access-249mq\") pod \"payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd\" (UID: \"2b60be98-7d0a-4e77-af20-5cf501487c84\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:13.595564 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.595412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b60be98-7d0a-4e77-af20-5cf501487c84-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd\" (UID: \"2b60be98-7d0a-4e77-af20-5cf501487c84\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:13.596166 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.596146    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b60be98-7d0a-4e77-af20-5cf501487c84-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd\" (UID: \"2b60be98-7d0a-4e77-af20-5cf501487c84\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:13.605194 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.605172    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-249mq\" (UniqueName: \"kubernetes.io/projected/2b60be98-7d0a-4e77-af20-5cf501487c84-kube-api-access-249mq\") pod \"payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd\" (UID: \"2b60be98-7d0a-4e77-af20-5cf501487c84\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:13.695491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.695462    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:13.970135 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:13.970075    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"]
Sep 11 18:50:13.972392 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:50:13.972368    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b60be98_7d0a_4e77_af20_5cf501487c84.slice/crio-04656471cb29510eb675c4be5bb2632dc81cededea44a2cd1ada48fcbeea182c WatchSource:0}: Error finding container 04656471cb29510eb675c4be5bb2632dc81cededea44a2cd1ada48fcbeea182c: Status 404 returned error can't find the container with id 04656471cb29510eb675c4be5bb2632dc81cededea44a2cd1ada48fcbeea182c
Sep 11 18:50:14.429415 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:14.429378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd" event={"ID":"2b60be98-7d0a-4e77-af20-5cf501487c84","Type":"ContainerStarted","Data":"9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b"}
Sep 11 18:50:14.429977 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:14.429437    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd" event={"ID":"2b60be98-7d0a-4e77-af20-5cf501487c84","Type":"ContainerStarted","Data":"04656471cb29510eb675c4be5bb2632dc81cededea44a2cd1ada48fcbeea182c"}
Sep 11 18:50:14.429977 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:14.429465    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:14.448345 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:14.448295    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd" podStartSLOduration=1.4482808710000001 podStartE2EDuration="1.448280871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:50:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:50:14.446184619 +0000 UTC m=+2077.481401290" watchObservedRunningTime="2026-09-11 18:50:14.448280871 +0000 UTC m=+2077.483497541"
Sep 11 18:50:21.545707 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.545628    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs" podUID="a9674211-9bc0-4c80-a680-d2a3aef41eed" containerName="main" containerID="cri-o://7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd" gracePeriod=45
Sep 11 18:50:21.842160 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.842134    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:50:21.873250 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873224    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9674211-9bc0-4c80-a680-d2a3aef41eed-tls-certs\") pod \"a9674211-9bc0-4c80-a680-d2a3aef41eed\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") "
Sep 11 18:50:21.873437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873272    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-home\") pod \"a9674211-9bc0-4c80-a680-d2a3aef41eed\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") "
Sep 11 18:50:21.873437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873321    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh6rw\" (UniqueName: \"kubernetes.io/projected/a9674211-9bc0-4c80-a680-d2a3aef41eed-kube-api-access-xh6rw\") pod \"a9674211-9bc0-4c80-a680-d2a3aef41eed\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") "
Sep 11 18:50:21.873437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873352    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-tmp-dir\") pod \"a9674211-9bc0-4c80-a680-d2a3aef41eed\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") "
Sep 11 18:50:21.873437 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873379    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-model-cache\") pod \"a9674211-9bc0-4c80-a680-d2a3aef41eed\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") "
Sep 11 18:50:21.873661 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873444    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-dshm\") pod \"a9674211-9bc0-4c80-a680-d2a3aef41eed\" (UID: \"a9674211-9bc0-4c80-a680-d2a3aef41eed\") "
Sep 11 18:50:21.873661 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a9674211-9bc0-4c80-a680-d2a3aef41eed" (UID: "a9674211-9bc0-4c80-a680-d2a3aef41eed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:50:21.873661 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873599    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-home" (OuterVolumeSpecName: "home") pod "a9674211-9bc0-4c80-a680-d2a3aef41eed" (UID: "a9674211-9bc0-4c80-a680-d2a3aef41eed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:50:21.873838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873651    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-model-cache" (OuterVolumeSpecName: "model-cache") pod "a9674211-9bc0-4c80-a680-d2a3aef41eed" (UID: "a9674211-9bc0-4c80-a680-d2a3aef41eed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:50:21.873893 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873840    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-home\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:50:21.873893 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873855    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-tmp-dir\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:50:21.873893 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.873865    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-model-cache\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:50:21.875493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.875462    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9674211-9bc0-4c80-a680-d2a3aef41eed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a9674211-9bc0-4c80-a680-d2a3aef41eed" (UID: "a9674211-9bc0-4c80-a680-d2a3aef41eed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:50:21.875493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.875483    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-dshm" (OuterVolumeSpecName: "dshm") pod "a9674211-9bc0-4c80-a680-d2a3aef41eed" (UID: "a9674211-9bc0-4c80-a680-d2a3aef41eed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:50:21.875914 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.875894    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9674211-9bc0-4c80-a680-d2a3aef41eed-kube-api-access-xh6rw" (OuterVolumeSpecName: "kube-api-access-xh6rw") pod "a9674211-9bc0-4c80-a680-d2a3aef41eed" (UID: "a9674211-9bc0-4c80-a680-d2a3aef41eed"). InnerVolumeSpecName "kube-api-access-xh6rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:50:21.974998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.974952    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9674211-9bc0-4c80-a680-d2a3aef41eed-tls-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:50:21.974998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.974992    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh6rw\" (UniqueName: \"kubernetes.io/projected/a9674211-9bc0-4c80-a680-d2a3aef41eed-kube-api-access-xh6rw\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:50:21.974998 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:21.975006    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9674211-9bc0-4c80-a680-d2a3aef41eed-dshm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:50:22.471932 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:22.471897    2578 generic.go:356] "Generic (PLEG): container finished" podID="a9674211-9bc0-4c80-a680-d2a3aef41eed" containerID="7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd" exitCode=0
Sep 11 18:50:22.472147 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:22.471941    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"
Sep 11 18:50:22.472147 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:22.471959    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs" event={"ID":"a9674211-9bc0-4c80-a680-d2a3aef41eed","Type":"ContainerDied","Data":"7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd"}
Sep 11 18:50:22.472147 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:22.471999    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs" event={"ID":"a9674211-9bc0-4c80-a680-d2a3aef41eed","Type":"ContainerDied","Data":"f2ecccff0d3f71e7e69e803bb1e796d56bd91a96e7cbee70ccb73bd35b12daf5"}
Sep 11 18:50:22.472147 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:22.472014    2578 scope.go:117] "RemoveContainer" containerID="7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd"
Sep 11 18:50:22.482004 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:22.481982    2578 scope.go:117] "RemoveContainer" containerID="7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd"
Sep 11 18:50:22.482308 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:50:22.482287    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd\": container with ID starting with 7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd not found: ID does not exist" containerID="7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd"
Sep 11 18:50:22.482392 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:22.482318    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd"} err="failed to get container status \"7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd\": rpc error: code = NotFound desc = could not find container \"7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd\": container with ID starting with 7952ad84cbecf04603d5c362c36507bd304c9939aae6a7b6481f2ea62dc105cd not found: ID does not exist"
Sep 11 18:50:22.493502 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:22.493477    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"]
Sep 11 18:50:22.497479 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:22.497459    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-553671/rate-test-model-553671-kserve-664bc6b7c5-vvfbs"]
Sep 11 18:50:23.411588 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:23.411545    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9674211-9bc0-4c80-a680-d2a3aef41eed" path="/var/lib/kubelet/pods/a9674211-9bc0-4c80-a680-d2a3aef41eed/volumes"
Sep 11 18:50:25.437613 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:25.437580    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:50:25.441831 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:25.441793    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 18:50:25.441968 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:25.441849    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 18:50:27.504618 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:27.504582    2578 generic.go:356] "Generic (PLEG): container finished" podID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerID="3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6" exitCode=0
Sep 11 18:50:27.505072 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:27.504630    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" event={"ID":"e49c860a-1dde-40bc-851d-9c5697f9684f","Type":"ContainerDied","Data":"3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6"}
Sep 11 18:50:27.505072 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:27.505039    2578 scope.go:117] "RemoveContainer" containerID="3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6"
Sep 11 18:50:27.506155 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:27.506124    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:27.506259 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:27.506210    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:27.506322 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:27.506256    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:28.514113 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:28.514070    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" event={"ID":"e49c860a-1dde-40bc-851d-9c5697f9684f","Type":"ContainerStarted","Data":"91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a"}
Sep 11 18:50:28.588697 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:28.588647    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:28.588920 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:28.588897    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 11 18:50:28.588973 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:28.588946    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 11 18:50:29.588894 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:29.588856    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 11 18:50:29.589289 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:29.588938    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 11 18:50:30.588539 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:30.588496    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 11 18:50:30.588755 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:30.588556    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 11 18:50:31.588980 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:31.588940    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 11 18:50:31.589396 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:31.589005    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 11 18:50:32.588545 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:32.588509    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 11 18:50:32.588759 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:32.588567    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 11 18:50:33.588527 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:33.588492    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Sep 11 18:50:33.588952 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:33.588548    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Sep 11 18:50:34.592220 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:34.592189    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:34.592635 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:34.592437    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:34.593246 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:34.593231    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:50:36.268573 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.268537    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"]
Sep 11 18:50:36.269137 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.269116    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9674211-9bc0-4c80-a680-d2a3aef41eed" containerName="main"
Sep 11 18:50:36.269137 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.269137    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9674211-9bc0-4c80-a680-d2a3aef41eed" containerName="main"
Sep 11 18:50:36.269291 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.269250    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9674211-9bc0-4c80-a680-d2a3aef41eed" containerName="main"
Sep 11 18:50:36.271480 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.271462    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.274013 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.273993    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-0ed2c8-openshift-default-dockercfg-g2dr9\""
Sep 11 18:50:36.285740 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.285713    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"]
Sep 11 18:50:36.412896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.412852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-socket\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.412896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.412903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-data\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.413165 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.412939    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-podinfo\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.413165 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.412997    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-token\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.413165 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.413042    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-envoy\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.413165 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.413093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-credential-socket\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.413165 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.413151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-certs\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.413442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.413175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istiod-ca-cert\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.413442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.413206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pfw7\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-kube-api-access-7pfw7\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514003 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.513961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-socket\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514003 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-data\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514263 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-podinfo\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514263 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-token\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514263 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-envoy\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514447 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514290    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-credential-socket\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514447 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-certs\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514563 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istiod-ca-cert\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514563 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514487    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pfw7\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-kube-api-access-7pfw7\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514563 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-socket\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514748 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514719    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-credential-socket\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514814 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-certs\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.514873 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.514837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-data\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.515170 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.515151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istiod-ca-cert\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.516595 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.516566    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-envoy\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.516738 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.516668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-podinfo\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.522388 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.522326    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pfw7\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-kube-api-access-7pfw7\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.522388 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.522343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-token\") pod \"e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") " pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.582931 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.582891    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:36.872766 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.869296    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"]
Sep 11 18:50:36.876319 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:50:36.876281    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a2c3a4_7346_402e_bb2e_f5acbf7fd8fc.slice/crio-76e6a99a4035565e6a6bb15aaf7926fc2401c2670466b2789057605e5c6afebe WatchSource:0}: Error finding container 76e6a99a4035565e6a6bb15aaf7926fc2401c2670466b2789057605e5c6afebe: Status 404 returned error can't find the container with id 76e6a99a4035565e6a6bb15aaf7926fc2401c2670466b2789057605e5c6afebe
Sep 11 18:50:36.878454 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.878425    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:36.878519 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.878484    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:36.878553 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:36.878525    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:37.559918 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:37.559885    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:50:37.565105 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:37.565082    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:50:37.569808 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:37.569785    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:50:37.574103 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:37.574077    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:50:37.576956 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:37.576925    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" event={"ID":"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc","Type":"ContainerStarted","Data":"bd0cecb54f404f24d4498b90b4c68a6f349ce00959437458a1934e77b65bab56"}
Sep 11 18:50:37.577055 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:37.576978    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" event={"ID":"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc","Type":"ContainerStarted","Data":"76e6a99a4035565e6a6bb15aaf7926fc2401c2670466b2789057605e5c6afebe"}
Sep 11 18:50:37.583453 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:37.583431    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:37.588497 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:37.588478    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:37.599608 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:37.599188    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" podStartSLOduration=1.5991697070000002 podStartE2EDuration="1.599169707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:50:37.597284831 +0000 UTC m=+2100.632501511" watchObservedRunningTime="2026-09-11 18:50:37.599169707 +0000 UTC m=+2100.634386379"
Sep 11 18:50:38.582665 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:38.582634    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:38.583787 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:38.583767    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:56.680857 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:56.680826    2578 generic.go:356] "Generic (PLEG): container finished" podID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerID="bd0cecb54f404f24d4498b90b4c68a6f349ce00959437458a1934e77b65bab56" exitCode=0
Sep 11 18:50:56.681325 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:56.680899    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" event={"ID":"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc","Type":"ContainerDied","Data":"bd0cecb54f404f24d4498b90b4c68a6f349ce00959437458a1934e77b65bab56"}
Sep 11 18:50:56.681472 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:56.681454    2578 scope.go:117] "RemoveContainer" containerID="bd0cecb54f404f24d4498b90b4c68a6f349ce00959437458a1934e77b65bab56"
Sep 11 18:50:56.682591 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:56.682565    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:56.682660 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:56.682633    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:56.682731 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:56.682662    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:50:57.691206 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:57.691166    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" event={"ID":"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc","Type":"ContainerStarted","Data":"d6e194a81d6d46225c98e91b27e8cfcebb12bfd7a5d2f77355045cced2d34803"}
Sep 11 18:50:58.583470 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:58.583427    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:50:58.583782 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:58.583756    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Sep 11 18:50:58.583839 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:58.583818    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Sep 11 18:50:59.583819 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:59.583781    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Sep 11 18:50:59.584233 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:50:59.583840    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Sep 11 18:51:00.583897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:00.583858    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Sep 11 18:51:00.584304 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:00.583917    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Sep 11 18:51:01.583643 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:01.583599    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Sep 11 18:51:01.583883 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:01.583663    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Sep 11 18:51:02.583672 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:02.583634    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Sep 11 18:51:02.584119 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:02.583733    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Sep 11 18:51:03.586911 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:03.586875    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:51:03.587385 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:03.587152    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:51:03.587788 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:03.587774    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:51:09.472327 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:09.472297    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lnhlh_7ddd784b-815b-42c0-bf68-667587dda11b/payload-processing/0.log"
Sep 11 18:51:37.893160 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:37.893086    2578 scope.go:117] "RemoveContainer" containerID="8073062dbdbf23b704390c632d1307d31da4a19eb91214a6c8e66625b46b460f"
Sep 11 18:51:37.901899 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:37.901879    2578 scope.go:117] "RemoveContainer" containerID="dd3dd3a53216bf7fb7edc6d71d83ae15ae01c2be40dca0b217909f2a48ffcbb5"
Sep 11 18:51:37.909787 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:37.909765    2578 scope.go:117] "RemoveContainer" containerID="2d4ee33ddf8599aa56cb2b983d51e124a7044fb1035c2a06d09484625d1beab8"
Sep 11 18:51:50.483949 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:50.483917    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd_2b60be98-7d0a-4e77-af20-5cf501487c84/payload-processing/0.log"
Sep 11 18:51:50.625601 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:50.625568    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lnhlh_7ddd784b-815b-42c0-bf68-667587dda11b/payload-processing/0.log"
Sep 11 18:51:58.663811 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.663776    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"]
Sep 11 18:51:58.667516 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.667495    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"]
Sep 11 18:51:58.667627 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.667608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:51:58.670061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.670039    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-1cb71b-dockercfg-mhw8t\""
Sep 11 18:51:58.670061 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.670052    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-1cb71b\""
Sep 11 18:51:58.670882 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.670863    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:51:58.678100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.678075    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"]
Sep 11 18:51:58.680151 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.680127    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"]
Sep 11 18:51:58.815039 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.814999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c\" (UID: \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:51:58.815219 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.815114    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nncrv\" (UniqueName: \"kubernetes.io/projected/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-kube-api-access-nncrv\") pod \"payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c\" (UID: \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:51:58.815219 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.815159    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2jcw\" (UniqueName: \"kubernetes.io/projected/42f0fb71-73de-4630-8d8d-67231cac69d2-kube-api-access-w2jcw\") pod \"payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86\" (UID: \"42f0fb71-73de-4630-8d8d-67231cac69d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:51:58.815316 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.815252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f0fb71-73de-4630-8d8d-67231cac69d2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86\" (UID: \"42f0fb71-73de-4630-8d8d-67231cac69d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:51:58.916336 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.916246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c\" (UID: \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:51:58.916336 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.916301    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nncrv\" (UniqueName: \"kubernetes.io/projected/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-kube-api-access-nncrv\") pod \"payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c\" (UID: \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:51:58.916336 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.916324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2jcw\" (UniqueName: \"kubernetes.io/projected/42f0fb71-73de-4630-8d8d-67231cac69d2-kube-api-access-w2jcw\") pod \"payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86\" (UID: \"42f0fb71-73de-4630-8d8d-67231cac69d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:51:58.916581 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.916372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f0fb71-73de-4630-8d8d-67231cac69d2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86\" (UID: \"42f0fb71-73de-4630-8d8d-67231cac69d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:51:58.917102 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.917082    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f0fb71-73de-4630-8d8d-67231cac69d2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86\" (UID: \"42f0fb71-73de-4630-8d8d-67231cac69d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:51:58.917154 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.917094    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c\" (UID: \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:51:58.924316 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.924292    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2jcw\" (UniqueName: \"kubernetes.io/projected/42f0fb71-73de-4630-8d8d-67231cac69d2-kube-api-access-w2jcw\") pod \"payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86\" (UID: \"42f0fb71-73de-4630-8d8d-67231cac69d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:51:58.924459 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.924320    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nncrv\" (UniqueName: \"kubernetes.io/projected/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-kube-api-access-nncrv\") pod \"payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c\" (UID: \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:51:58.981424 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.981394    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:51:58.988513 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:58.988495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:51:59.258129 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:59.258054    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"]
Sep 11 18:51:59.261050 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:51:59.261021    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89b7d90_fdc0_4599_b9d8_ba3749ba5cdc.slice/crio-6078d17aef87233a43e8d9c6c7ef12159dcc471209c1f270224365d6e7310806 WatchSource:0}: Error finding container 6078d17aef87233a43e8d9c6c7ef12159dcc471209c1f270224365d6e7310806: Status 404 returned error can't find the container with id 6078d17aef87233a43e8d9c6c7ef12159dcc471209c1f270224365d6e7310806
Sep 11 18:51:59.269034 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:51:59.268989    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"]
Sep 11 18:51:59.270121 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:51:59.270098    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f0fb71_73de_4630_8d8d_67231cac69d2.slice/crio-149ffcfd219d1b4be65af7bb8a654a1c2674a6f1cdf42245d0cea461e4722196 WatchSource:0}: Error finding container 149ffcfd219d1b4be65af7bb8a654a1c2674a6f1cdf42245d0cea461e4722196: Status 404 returned error can't find the container with id 149ffcfd219d1b4be65af7bb8a654a1c2674a6f1cdf42245d0cea461e4722196
Sep 11 18:52:00.049736 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:00.049701    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86" event={"ID":"42f0fb71-73de-4630-8d8d-67231cac69d2","Type":"ContainerStarted","Data":"4ea45e4865afcf8b05a2b89c070a49ecca7123e20ef1efe277bfccdfdf1876c8"}
Sep 11 18:52:00.050210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:00.049765    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:52:00.050210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:00.049790    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86" event={"ID":"42f0fb71-73de-4630-8d8d-67231cac69d2","Type":"ContainerStarted","Data":"149ffcfd219d1b4be65af7bb8a654a1c2674a6f1cdf42245d0cea461e4722196"}
Sep 11 18:52:00.051532 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:00.051495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c" event={"ID":"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc","Type":"ContainerStarted","Data":"a40b12852af484b2e489f9c60db6f126ea240dc24c0a377688f8714c25b7759f"}
Sep 11 18:52:00.051641 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:00.051548    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c" event={"ID":"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc","Type":"ContainerStarted","Data":"6078d17aef87233a43e8d9c6c7ef12159dcc471209c1f270224365d6e7310806"}
Sep 11 18:52:00.051641 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:00.051579    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:52:00.065137 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:00.064921    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86" podStartSLOduration=2.064903203 podStartE2EDuration="2.064903203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:52:00.064897413 +0000 UTC m=+2183.100114084" watchObservedRunningTime="2026-09-11 18:52:00.064903203 +0000 UTC m=+2183.100119871"
Sep 11 18:52:00.080798 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:00.080752    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c" podStartSLOduration=2.080733715 podStartE2EDuration="2.080733715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:52:00.079264549 +0000 UTC m=+2183.114481231" watchObservedRunningTime="2026-09-11 18:52:00.080733715 +0000 UTC m=+2183.115950386"
Sep 11 18:52:11.058855 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:11.058824    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:52:11.059396 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:11.058898    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:52:25.460745 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:25.460711    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"]
Sep 11 18:52:25.463153 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:25.463134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"
Sep 11 18:52:25.465594 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:25.465567    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:52:25.465594 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:25.465573    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:52:25.471735 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:25.471709    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"]
Sep 11 18:52:25.575353 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:25.575320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50787d92-fa13-4e04-a521-a9277c7e4038-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6\" (UID: \"50787d92-fa13-4e04-a521-a9277c7e4038\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"
Sep 11 18:52:25.676655 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:25.676615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50787d92-fa13-4e04-a521-a9277c7e4038-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6\" (UID: \"50787d92-fa13-4e04-a521-a9277c7e4038\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"
Sep 11 18:52:25.677286 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:25.677266    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50787d92-fa13-4e04-a521-a9277c7e4038-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6\" (UID: \"50787d92-fa13-4e04-a521-a9277c7e4038\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"
Sep 11 18:52:25.775203 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:25.775116    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"
Sep 11 18:52:26.044713 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:26.044689    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"]
Sep 11 18:52:26.047045 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:52:26.047016    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50787d92_fa13_4e04_a521_a9277c7e4038.slice/crio-0a6575043c8f040f24e9c0bd71dc24b18a55edef8c1bb9ff49e4d12b7df273ef WatchSource:0}: Error finding container 0a6575043c8f040f24e9c0bd71dc24b18a55edef8c1bb9ff49e4d12b7df273ef: Status 404 returned error can't find the container with id 0a6575043c8f040f24e9c0bd71dc24b18a55edef8c1bb9ff49e4d12b7df273ef
Sep 11 18:52:26.195138 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:26.195103    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6" event={"ID":"50787d92-fa13-4e04-a521-a9277c7e4038","Type":"ContainerStarted","Data":"0a6575043c8f040f24e9c0bd71dc24b18a55edef8c1bb9ff49e4d12b7df273ef"}
Sep 11 18:52:27.202249 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:27.202213    2578 generic.go:356] "Generic (PLEG): container finished" podID="50787d92-fa13-4e04-a521-a9277c7e4038" containerID="56209aee40ad4511bfcc01af5d65edc86f9c366c0b4657418b7fe380522afafa" exitCode=0
Sep 11 18:52:27.202714 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:27.202306    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6" event={"ID":"50787d92-fa13-4e04-a521-a9277c7e4038","Type":"ContainerDied","Data":"56209aee40ad4511bfcc01af5d65edc86f9c366c0b4657418b7fe380522afafa"}
Sep 11 18:52:28.346040 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:28.346013    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"
Sep 11 18:52:28.401527 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:28.401500    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50787d92-fa13-4e04-a521-a9277c7e4038-maas-api-service-ca\") pod \"50787d92-fa13-4e04-a521-a9277c7e4038\" (UID: \"50787d92-fa13-4e04-a521-a9277c7e4038\") "
Sep 11 18:52:28.401909 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:28.401885    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50787d92-fa13-4e04-a521-a9277c7e4038-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50787d92-fa13-4e04-a521-a9277c7e4038" (UID: "50787d92-fa13-4e04-a521-a9277c7e4038"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:52:28.503118 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:28.503030    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50787d92-fa13-4e04-a521-a9277c7e4038-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:52:29.218698 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:29.218645    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6" event={"ID":"50787d92-fa13-4e04-a521-a9277c7e4038","Type":"ContainerDied","Data":"0a6575043c8f040f24e9c0bd71dc24b18a55edef8c1bb9ff49e4d12b7df273ef"}
Sep 11 18:52:29.218886 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:29.218732    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a6575043c8f040f24e9c0bd71dc24b18a55edef8c1bb9ff49e4d12b7df273ef"
Sep 11 18:52:29.218886 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:29.218661    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"
Sep 11 18:52:36.986840 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:36.986800    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"]
Sep 11 18:52:36.987325 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:36.987011    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86" podUID="42f0fb71-73de-4630-8d8d-67231cac69d2" containerName="payload-processing" containerID="cri-o://4ea45e4865afcf8b05a2b89c070a49ecca7123e20ef1efe277bfccdfdf1876c8" gracePeriod=30
Sep 11 18:52:37.012498 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.012468    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"]
Sep 11 18:52:37.012728 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.012669    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c" podUID="a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc" containerName="payload-pre-processing" containerID="cri-o://a40b12852af484b2e489f9c60db6f126ea240dc24c0a377688f8714c25b7759f" gracePeriod=30
Sep 11 18:52:37.281397 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.279779    2578 generic.go:356] "Generic (PLEG): container finished" podID="a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc" containerID="a40b12852af484b2e489f9c60db6f126ea240dc24c0a377688f8714c25b7759f" exitCode=0
Sep 11 18:52:37.281397 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.279980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c" event={"ID":"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc","Type":"ContainerDied","Data":"a40b12852af484b2e489f9c60db6f126ea240dc24c0a377688f8714c25b7759f"}
Sep 11 18:52:37.287663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.284191    2578 generic.go:356] "Generic (PLEG): container finished" podID="42f0fb71-73de-4630-8d8d-67231cac69d2" containerID="4ea45e4865afcf8b05a2b89c070a49ecca7123e20ef1efe277bfccdfdf1876c8" exitCode=0
Sep 11 18:52:37.287663 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.284344    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86" event={"ID":"42f0fb71-73de-4630-8d8d-67231cac69d2","Type":"ContainerDied","Data":"4ea45e4865afcf8b05a2b89c070a49ecca7123e20ef1efe277bfccdfdf1876c8"}
Sep 11 18:52:37.314347 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.314321    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:52:37.317493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.317471    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:52:37.388860 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.388829    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2jcw\" (UniqueName: \"kubernetes.io/projected/42f0fb71-73de-4630-8d8d-67231cac69d2-kube-api-access-w2jcw\") pod \"42f0fb71-73de-4630-8d8d-67231cac69d2\" (UID: \"42f0fb71-73de-4630-8d8d-67231cac69d2\") "
Sep 11 18:52:37.389049 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.388891    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-plugins-config-volume\") pod \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\" (UID: \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\") "
Sep 11 18:52:37.389049 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.388954    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f0fb71-73de-4630-8d8d-67231cac69d2-plugins-config-volume\") pod \"42f0fb71-73de-4630-8d8d-67231cac69d2\" (UID: \"42f0fb71-73de-4630-8d8d-67231cac69d2\") "
Sep 11 18:52:37.389049 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.389018    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nncrv\" (UniqueName: \"kubernetes.io/projected/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-kube-api-access-nncrv\") pod \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\" (UID: \"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc\") "
Sep 11 18:52:37.389339 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.389309    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc" (UID: "a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:52:37.389399 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.389360    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42f0fb71-73de-4630-8d8d-67231cac69d2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "42f0fb71-73de-4630-8d8d-67231cac69d2" (UID: "42f0fb71-73de-4630-8d8d-67231cac69d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:52:37.391024 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.390999    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-kube-api-access-nncrv" (OuterVolumeSpecName: "kube-api-access-nncrv") pod "a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc" (UID: "a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc"). InnerVolumeSpecName "kube-api-access-nncrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:52:37.391110 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.391047    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f0fb71-73de-4630-8d8d-67231cac69d2-kube-api-access-w2jcw" (OuterVolumeSpecName: "kube-api-access-w2jcw") pod "42f0fb71-73de-4630-8d8d-67231cac69d2" (UID: "42f0fb71-73de-4630-8d8d-67231cac69d2"). InnerVolumeSpecName "kube-api-access-w2jcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:52:37.489861 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.489822    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nncrv\" (UniqueName: \"kubernetes.io/projected/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-kube-api-access-nncrv\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:52:37.489861 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.489854    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2jcw\" (UniqueName: \"kubernetes.io/projected/42f0fb71-73de-4630-8d8d-67231cac69d2-kube-api-access-w2jcw\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:52:37.489861 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.489864    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:52:37.490112 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.489873    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f0fb71-73de-4630-8d8d-67231cac69d2-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:52:37.504517 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.504481    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"]
Sep 11 18:52:37.513082 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.513052    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-1cb71b-t2dx6"]
Sep 11 18:52:37.937540 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.937508    2578 scope.go:117] "RemoveContainer" containerID="055c366e15454d4d6e683348dd96b5aabe0ce0fbee44f03c7a65ea614f07c0f7"
Sep 11 18:52:37.946198 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.946174    2578 scope.go:117] "RemoveContainer" containerID="2994dfc0ac950521f9852e62ccc2c4f87b6354aa5f3096e23d59bcef062434fe"
Sep 11 18:52:37.954210 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.954188    2578 scope.go:117] "RemoveContainer" containerID="b36824ce93f0da08346716554321a03550957afb497b4878816f5b024f3d34ec"
Sep 11 18:52:37.962276 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:37.962257    2578 scope.go:117] "RemoveContainer" containerID="df696a7e26dd47a4c0ea89265cadd53e65ce50e6748246f69ff800b563525399"
Sep 11 18:52:38.291638 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.291543    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"
Sep 11 18:52:38.291638 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.291568    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86" event={"ID":"42f0fb71-73de-4630-8d8d-67231cac69d2","Type":"ContainerDied","Data":"149ffcfd219d1b4be65af7bb8a654a1c2674a6f1cdf42245d0cea461e4722196"}
Sep 11 18:52:38.292183 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.291702    2578 scope.go:117] "RemoveContainer" containerID="4ea45e4865afcf8b05a2b89c070a49ecca7123e20ef1efe277bfccdfdf1876c8"
Sep 11 18:52:38.293117 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.293096    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"
Sep 11 18:52:38.293250 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.293086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c" event={"ID":"a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc","Type":"ContainerDied","Data":"6078d17aef87233a43e8d9c6c7ef12159dcc471209c1f270224365d6e7310806"}
Sep 11 18:52:38.303983 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.303948    2578 scope.go:117] "RemoveContainer" containerID="a40b12852af484b2e489f9c60db6f126ea240dc24c0a377688f8714c25b7759f"
Sep 11 18:52:38.336302 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.336259    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"]
Sep 11 18:52:38.346877 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.346840    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1cb71b-77cd48c4ff-2wv86"]
Sep 11 18:52:38.378457 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.378419    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"]
Sep 11 18:52:38.386605 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:38.386571    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1cb71b-7895b884c8-8kh2c"]
Sep 11 18:52:39.410722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:39.410693    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42f0fb71-73de-4630-8d8d-67231cac69d2" path="/var/lib/kubelet/pods/42f0fb71-73de-4630-8d8d-67231cac69d2/volumes"
Sep 11 18:52:39.411113 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:39.411053    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50787d92-fa13-4e04-a521-a9277c7e4038" path="/var/lib/kubelet/pods/50787d92-fa13-4e04-a521-a9277c7e4038/volumes"
Sep 11 18:52:39.411507 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:39.411490    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc" path="/var/lib/kubelet/pods/a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc/volumes"
Sep 11 18:52:48.374899 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.374865    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"]
Sep 11 18:52:48.375312 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.375302    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc" containerName="payload-pre-processing"
Sep 11 18:52:48.375354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.375315    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc" containerName="payload-pre-processing"
Sep 11 18:52:48.375354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.375323    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f0fb71-73de-4630-8d8d-67231cac69d2" containerName="payload-processing"
Sep 11 18:52:48.375354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.375330    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f0fb71-73de-4630-8d8d-67231cac69d2" containerName="payload-processing"
Sep 11 18:52:48.375354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.375344    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50787d92-fa13-4e04-a521-a9277c7e4038" containerName="revoke-keys"
Sep 11 18:52:48.375354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.375350    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50787d92-fa13-4e04-a521-a9277c7e4038" containerName="revoke-keys"
Sep 11 18:52:48.375511 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.375404    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f0fb71-73de-4630-8d8d-67231cac69d2" containerName="payload-processing"
Sep 11 18:52:48.375511 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.375415    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a89b7d90-fdc0-4599-b9d8-ba3749ba5cdc" containerName="payload-pre-processing"
Sep 11 18:52:48.375511 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.375427    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="50787d92-fa13-4e04-a521-a9277c7e4038" containerName="revoke-keys"
Sep 11 18:52:48.378590 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.378574    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"
Sep 11 18:52:48.381283 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.381254    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:52:48.381405 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.381286    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:52:48.392518 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.392495    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"]
Sep 11 18:52:48.498056 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.498021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38022589-38c7-4670-8390-78442ce0ef51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb\" (UID: \"38022589-38c7-4670-8390-78442ce0ef51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"
Sep 11 18:52:48.598745 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.598706    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38022589-38c7-4670-8390-78442ce0ef51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb\" (UID: \"38022589-38c7-4670-8390-78442ce0ef51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"
Sep 11 18:52:48.599303 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.599282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38022589-38c7-4670-8390-78442ce0ef51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb\" (UID: \"38022589-38c7-4670-8390-78442ce0ef51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"
Sep 11 18:52:48.689298 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.689264    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"
Sep 11 18:52:48.967127 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:48.967098    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"]
Sep 11 18:52:48.969495 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:52:48.969460    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38022589_38c7_4670_8390_78442ce0ef51.slice/crio-843cc9fcd3eb62b9883bd6c9043bb960713e68b4139e91297ae066ba65651778 WatchSource:0}: Error finding container 843cc9fcd3eb62b9883bd6c9043bb960713e68b4139e91297ae066ba65651778: Status 404 returned error can't find the container with id 843cc9fcd3eb62b9883bd6c9043bb960713e68b4139e91297ae066ba65651778
Sep 11 18:52:49.364613 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:49.364574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb" event={"ID":"38022589-38c7-4670-8390-78442ce0ef51","Type":"ContainerStarted","Data":"f4260afc919fce29a0d53ae62bd43072d75bff1f47b9ee224a3ac43eda94b64f"}
Sep 11 18:52:49.364836 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:49.364637    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb" event={"ID":"38022589-38c7-4670-8390-78442ce0ef51","Type":"ContainerStarted","Data":"843cc9fcd3eb62b9883bd6c9043bb960713e68b4139e91297ae066ba65651778"}
Sep 11 18:52:49.381644 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:49.381595    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb" podStartSLOduration=1.381579956 podStartE2EDuration="1.381579956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:52:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:52:49.379637823 +0000 UTC m=+2232.414854493" watchObservedRunningTime="2026-09-11 18:52:49.381579956 +0000 UTC m=+2232.416796626"
Sep 11 18:52:50.372161 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:50.372129    2578 generic.go:356] "Generic (PLEG): container finished" podID="38022589-38c7-4670-8390-78442ce0ef51" containerID="f4260afc919fce29a0d53ae62bd43072d75bff1f47b9ee224a3ac43eda94b64f" exitCode=0
Sep 11 18:52:50.372340 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:50.372193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb" event={"ID":"38022589-38c7-4670-8390-78442ce0ef51","Type":"ContainerDied","Data":"f4260afc919fce29a0d53ae62bd43072d75bff1f47b9ee224a3ac43eda94b64f"}
Sep 11 18:52:51.508456 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:51.508431    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"
Sep 11 18:52:51.625816 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:51.625786    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38022589-38c7-4670-8390-78442ce0ef51-maas-api-service-ca\") pod \"38022589-38c7-4670-8390-78442ce0ef51\" (UID: \"38022589-38c7-4670-8390-78442ce0ef51\") "
Sep 11 18:52:51.626178 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:51.626158    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38022589-38c7-4670-8390-78442ce0ef51-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38022589-38c7-4670-8390-78442ce0ef51" (UID: "38022589-38c7-4670-8390-78442ce0ef51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:52:51.726923 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:51.726887    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38022589-38c7-4670-8390-78442ce0ef51-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:52:52.384030 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:52.383992    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb" event={"ID":"38022589-38c7-4670-8390-78442ce0ef51","Type":"ContainerDied","Data":"843cc9fcd3eb62b9883bd6c9043bb960713e68b4139e91297ae066ba65651778"}
Sep 11 18:52:52.384215 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:52.384031    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"
Sep 11 18:52:52.384215 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:52:52.384057    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="843cc9fcd3eb62b9883bd6c9043bb960713e68b4139e91297ae066ba65651778"
Sep 11 18:53:03.997690 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:03.997583    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"]
Sep 11 18:53:03.998296 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:03.997817    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd" podUID="2b60be98-7d0a-4e77-af20-5cf501487c84" containerName="payload-processing" containerID="cri-o://9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b" gracePeriod=30
Sep 11 18:53:04.188455 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.188421    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"]
Sep 11 18:53:04.191806 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.191769    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-da4d13-sdfxb"]
Sep 11 18:53:04.259211 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.259140    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:53:04.352636 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.350742    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b60be98-7d0a-4e77-af20-5cf501487c84-plugins-config-volume\") pod \"2b60be98-7d0a-4e77-af20-5cf501487c84\" (UID: \"2b60be98-7d0a-4e77-af20-5cf501487c84\") "
Sep 11 18:53:04.352636 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.350856    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-249mq\" (UniqueName: \"kubernetes.io/projected/2b60be98-7d0a-4e77-af20-5cf501487c84-kube-api-access-249mq\") pod \"2b60be98-7d0a-4e77-af20-5cf501487c84\" (UID: \"2b60be98-7d0a-4e77-af20-5cf501487c84\") "
Sep 11 18:53:04.352636 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.351164    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b60be98-7d0a-4e77-af20-5cf501487c84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b60be98-7d0a-4e77-af20-5cf501487c84" (UID: "2b60be98-7d0a-4e77-af20-5cf501487c84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:53:04.357153 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.354897    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b60be98-7d0a-4e77-af20-5cf501487c84-kube-api-access-249mq" (OuterVolumeSpecName: "kube-api-access-249mq") pod "2b60be98-7d0a-4e77-af20-5cf501487c84" (UID: "2b60be98-7d0a-4e77-af20-5cf501487c84"). InnerVolumeSpecName "kube-api-access-249mq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:53:04.448257 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.448226    2578 generic.go:356] "Generic (PLEG): container finished" podID="2b60be98-7d0a-4e77-af20-5cf501487c84" containerID="9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b" exitCode=0
Sep 11 18:53:04.448443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.448273    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"
Sep 11 18:53:04.448443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.448314    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd" event={"ID":"2b60be98-7d0a-4e77-af20-5cf501487c84","Type":"ContainerDied","Data":"9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b"}
Sep 11 18:53:04.448443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.448365    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd" event={"ID":"2b60be98-7d0a-4e77-af20-5cf501487c84","Type":"ContainerDied","Data":"04656471cb29510eb675c4be5bb2632dc81cededea44a2cd1ada48fcbeea182c"}
Sep 11 18:53:04.448443 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.448390    2578 scope.go:117] "RemoveContainer" containerID="9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b"
Sep 11 18:53:04.451842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.451817    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b60be98-7d0a-4e77-af20-5cf501487c84-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:53:04.451842 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.451844    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-249mq\" (UniqueName: \"kubernetes.io/projected/2b60be98-7d0a-4e77-af20-5cf501487c84-kube-api-access-249mq\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:53:04.458403 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.458387    2578 scope.go:117] "RemoveContainer" containerID="9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b"
Sep 11 18:53:04.458668 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:53:04.458648    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b\": container with ID starting with 9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b not found: ID does not exist" containerID="9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b"
Sep 11 18:53:04.458750 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.458707    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b"} err="failed to get container status \"9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b\": rpc error: code = NotFound desc = could not find container \"9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b\": container with ID starting with 9280b856de86e4926e6824b0bb91eb15ec8b4993898b5546932bfb6c551b0a9b not found: ID does not exist"
Sep 11 18:53:04.471550 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.471519    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"]
Sep 11 18:53:04.473486 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:04.473466    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-da4d13-56fd97c675-2fdgd"]
Sep 11 18:53:05.410472 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:05.410443    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b60be98-7d0a-4e77-af20-5cf501487c84" path="/var/lib/kubelet/pods/2b60be98-7d0a-4e77-af20-5cf501487c84/volumes"
Sep 11 18:53:05.410896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:05.410800    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38022589-38c7-4670-8390-78442ce0ef51" path="/var/lib/kubelet/pods/38022589-38c7-4670-8390-78442ce0ef51/volumes"
Sep 11 18:53:58.753606 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:58.753566    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"]
Sep 11 18:53:58.754171 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:58.753901    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy" containerID="cri-o://91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a" gracePeriod=30
Sep 11 18:53:59.141390 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.141311    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"]
Sep 11 18:53:59.141701 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.141672    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b60be98-7d0a-4e77-af20-5cf501487c84" containerName="payload-processing"
Sep 11 18:53:59.141701 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.141699    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b60be98-7d0a-4e77-af20-5cf501487c84" containerName="payload-processing"
Sep 11 18:53:59.141787 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.141716    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38022589-38c7-4670-8390-78442ce0ef51" containerName="revoke-keys"
Sep 11 18:53:59.141787 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.141722    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="38022589-38c7-4670-8390-78442ce0ef51" containerName="revoke-keys"
Sep 11 18:53:59.141852 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.141788    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="38022589-38c7-4670-8390-78442ce0ef51" containerName="revoke-keys"
Sep 11 18:53:59.141852 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.141795    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b60be98-7d0a-4e77-af20-5cf501487c84" containerName="payload-processing"
Sep 11 18:53:59.144854 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.144837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"
Sep 11 18:53:59.147317 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.147291    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:53:59.147439 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.147300    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:53:59.151922 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.151894    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"]
Sep 11 18:53:59.290889 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.290859    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f409f6c-7565-43f1-b480-768c58053de7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq\" (UID: \"5f409f6c-7565-43f1-b480-768c58053de7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"
Sep 11 18:53:59.392361 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.392280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f409f6c-7565-43f1-b480-768c58053de7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq\" (UID: \"5f409f6c-7565-43f1-b480-768c58053de7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"
Sep 11 18:53:59.392914 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.392893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f409f6c-7565-43f1-b480-768c58053de7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq\" (UID: \"5f409f6c-7565-43f1-b480-768c58053de7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"
Sep 11 18:53:59.455886 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.455854    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"
Sep 11 18:53:59.726375 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:53:59.726353    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"]
Sep 11 18:53:59.728439 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:53:59.728411    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f409f6c_7565_43f1_b480_768c58053de7.slice/crio-d4f82c3bd8418745261982466341fd741d6ba307b16bed0f6ceecce60d0c5f9e WatchSource:0}: Error finding container d4f82c3bd8418745261982466341fd741d6ba307b16bed0f6ceecce60d0c5f9e: Status 404 returned error can't find the container with id d4f82c3bd8418745261982466341fd741d6ba307b16bed0f6ceecce60d0c5f9e
Sep 11 18:54:00.738237 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:00.738195    2578 generic.go:356] "Generic (PLEG): container finished" podID="5f409f6c-7565-43f1-b480-768c58053de7" containerID="607e59905ba0cfff5be2a2252dfc782bd164c50efa5e11e4dc8c909dfe056b06" exitCode=0
Sep 11 18:54:00.738733 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:00.738286    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq" event={"ID":"5f409f6c-7565-43f1-b480-768c58053de7","Type":"ContainerDied","Data":"607e59905ba0cfff5be2a2252dfc782bd164c50efa5e11e4dc8c909dfe056b06"}
Sep 11 18:54:00.738733 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:00.738335    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq" event={"ID":"5f409f6c-7565-43f1-b480-768c58053de7","Type":"ContainerStarted","Data":"d4f82c3bd8418745261982466341fd741d6ba307b16bed0f6ceecce60d0c5f9e"}
Sep 11 18:54:01.875974 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:01.875945    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"
Sep 11 18:54:02.018622 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:02.018526    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f409f6c-7565-43f1-b480-768c58053de7-maas-api-service-ca\") pod \"5f409f6c-7565-43f1-b480-768c58053de7\" (UID: \"5f409f6c-7565-43f1-b480-768c58053de7\") "
Sep 11 18:54:02.018901 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:02.018876    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f409f6c-7565-43f1-b480-768c58053de7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f409f6c-7565-43f1-b480-768c58053de7" (UID: "5f409f6c-7565-43f1-b480-768c58053de7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:54:02.119238 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:02.119203    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f409f6c-7565-43f1-b480-768c58053de7-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:02.751591 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:02.751563    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"
Sep 11 18:54:02.751798 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:02.751607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq" event={"ID":"5f409f6c-7565-43f1-b480-768c58053de7","Type":"ContainerDied","Data":"d4f82c3bd8418745261982466341fd741d6ba307b16bed0f6ceecce60d0c5f9e"}
Sep 11 18:54:02.751798 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:02.751652    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4f82c3bd8418745261982466341fd741d6ba307b16bed0f6ceecce60d0c5f9e"
Sep 11 18:54:03.995709 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:03.995665    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:54:04.139103 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139017    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-credential-socket\") pod \"e49c860a-1dde-40bc-851d-9c5697f9684f\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") "
Sep 11 18:54:04.139103 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139055    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t42v\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-kube-api-access-7t42v\") pod \"e49c860a-1dde-40bc-851d-9c5697f9684f\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") "
Sep 11 18:54:04.139103 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139084    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-socket\") pod \"e49c860a-1dde-40bc-851d-9c5697f9684f\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") "
Sep 11 18:54:04.139379 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139113    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-podinfo\") pod \"e49c860a-1dde-40bc-851d-9c5697f9684f\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") "
Sep 11 18:54:04.139379 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139258    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-envoy\") pod \"e49c860a-1dde-40bc-851d-9c5697f9684f\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") "
Sep 11 18:54:04.139379 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139328    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e49c860a-1dde-40bc-851d-9c5697f9684f" (UID: "e49c860a-1dde-40bc-851d-9c5697f9684f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:04.139379 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139355    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-certs\") pod \"e49c860a-1dde-40bc-851d-9c5697f9684f\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") "
Sep 11 18:54:04.139576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139423    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49c860a-1dde-40bc-851d-9c5697f9684f-istiod-ca-cert\") pod \"e49c860a-1dde-40bc-851d-9c5697f9684f\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") "
Sep 11 18:54:04.139576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139450    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-data\") pod \"e49c860a-1dde-40bc-851d-9c5697f9684f\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") "
Sep 11 18:54:04.139576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139506    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e49c860a-1dde-40bc-851d-9c5697f9684f" (UID: "e49c860a-1dde-40bc-851d-9c5697f9684f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:04.139576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139528    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e49c860a-1dde-40bc-851d-9c5697f9684f" (UID: "e49c860a-1dde-40bc-851d-9c5697f9684f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:04.139576 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139537    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-token\") pod \"e49c860a-1dde-40bc-851d-9c5697f9684f\" (UID: \"e49c860a-1dde-40bc-851d-9c5697f9684f\") "
Sep 11 18:54:04.139869 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139738    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e49c860a-1dde-40bc-851d-9c5697f9684f" (UID: "e49c860a-1dde-40bc-851d-9c5697f9684f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:04.139869 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139771    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49c860a-1dde-40bc-851d-9c5697f9684f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e49c860a-1dde-40bc-851d-9c5697f9684f" (UID: "e49c860a-1dde-40bc-851d-9c5697f9684f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:54:04.140026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.139987    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:04.140087 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.140030    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49c860a-1dde-40bc-851d-9c5697f9684f-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:04.140087 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.140043    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:04.140087 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.140057    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:04.140087 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.140071    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:04.141507 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.141480    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e49c860a-1dde-40bc-851d-9c5697f9684f" (UID: "e49c860a-1dde-40bc-851d-9c5697f9684f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:54:04.141618 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.141528    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e49c860a-1dde-40bc-851d-9c5697f9684f" (UID: "e49c860a-1dde-40bc-851d-9c5697f9684f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:04.141618 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.141542    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-kube-api-access-7t42v" (OuterVolumeSpecName: "kube-api-access-7t42v") pod "e49c860a-1dde-40bc-851d-9c5697f9684f" (UID: "e49c860a-1dde-40bc-851d-9c5697f9684f"). InnerVolumeSpecName "kube-api-access-7t42v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:54:04.141731 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.141616    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e49c860a-1dde-40bc-851d-9c5697f9684f" (UID: "e49c860a-1dde-40bc-851d-9c5697f9684f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:54:04.240936 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.240905    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:04.240936 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.240932    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t42v\" (UniqueName: \"kubernetes.io/projected/e49c860a-1dde-40bc-851d-9c5697f9684f-kube-api-access-7t42v\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:04.240936 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.240942    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:04.241138 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.240950    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49c860a-1dde-40bc-851d-9c5697f9684f-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:04.763817 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.763784    2578 generic.go:356] "Generic (PLEG): container finished" podID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerID="91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a" exitCode=0
Sep 11 18:54:04.764021 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.763837    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"
Sep 11 18:54:04.764021 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.763845    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" event={"ID":"e49c860a-1dde-40bc-851d-9c5697f9684f","Type":"ContainerDied","Data":"91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a"}
Sep 11 18:54:04.764021 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.763899    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm" event={"ID":"e49c860a-1dde-40bc-851d-9c5697f9684f","Type":"ContainerDied","Data":"131f16a282c9e8f74cf080a5554b615076aa7278f3b418c66def66c4c0b74a38"}
Sep 11 18:54:04.764021 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.763929    2578 scope.go:117] "RemoveContainer" containerID="91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a"
Sep 11 18:54:04.773414 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.773395    2578 scope.go:117] "RemoveContainer" containerID="3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6"
Sep 11 18:54:04.781490 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.781472    2578 scope.go:117] "RemoveContainer" containerID="91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a"
Sep 11 18:54:04.781770 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:54:04.781751    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a\": container with ID starting with 91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a not found: ID does not exist" containerID="91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a"
Sep 11 18:54:04.781821 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.781781    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a"} err="failed to get container status \"91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a\": rpc error: code = NotFound desc = could not find container \"91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a\": container with ID starting with 91f2d94a05ca71c8cb23d4b062a1945fc80552ebedb4a1730c993c57a8b98d8a not found: ID does not exist"
Sep 11 18:54:04.781821 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.781798    2578 scope.go:117] "RemoveContainer" containerID="3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6"
Sep 11 18:54:04.782058 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:54:04.782039    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6\": container with ID starting with 3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6 not found: ID does not exist" containerID="3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6"
Sep 11 18:54:04.782117 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.782064    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6"} err="failed to get container status \"3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6\": rpc error: code = NotFound desc = could not find container \"3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6\": container with ID starting with 3da6654667c880deacc5d14467ef7dc61174ccc85b35c7298401240e0d121df6 not found: ID does not exist"
Sep 11 18:54:04.787198 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.787177    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"]
Sep 11 18:54:04.794271 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:04.794248    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-da4d13-openshift-default-78885dfd96-hhjpm"]
Sep 11 18:54:05.409418 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:05.409384    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" path="/var/lib/kubelet/pods/e49c860a-1dde-40bc-851d-9c5697f9684f/volumes"
Sep 11 18:54:10.049892 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:10.049854    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"]
Sep 11 18:54:10.054517 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:10.054491    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0ed2c8-x97vq"]
Sep 11 18:54:11.409336 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:11.409303    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f409f6c-7565-43f1-b480-768c58053de7" path="/var/lib/kubelet/pods/5f409f6c-7565-43f1-b480-768c58053de7/volumes"
Sep 11 18:54:16.672069 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:16.672034    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"]
Sep 11 18:54:16.672439 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:16.672326    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy" containerID="cri-o://d6e194a81d6d46225c98e91b27e8cfcebb12bfd7a5d2f77355045cced2d34803" gracePeriod=30
Sep 11 18:54:17.066882 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.066795    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"]
Sep 11 18:54:17.067172 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.067159    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy"
Sep 11 18:54:17.067225 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.067173    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy"
Sep 11 18:54:17.067225 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.067184    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f409f6c-7565-43f1-b480-768c58053de7" containerName="revoke-keys"
Sep 11 18:54:17.067225 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.067190    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f409f6c-7565-43f1-b480-768c58053de7" containerName="revoke-keys"
Sep 11 18:54:17.067225 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.067199    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy"
Sep 11 18:54:17.067225 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.067205    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy"
Sep 11 18:54:17.067404 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.067270    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy"
Sep 11 18:54:17.067404 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.067278    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f409f6c-7565-43f1-b480-768c58053de7" containerName="revoke-keys"
Sep 11 18:54:17.067404 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.067283    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49c860a-1dde-40bc-851d-9c5697f9684f" containerName="istio-proxy"
Sep 11 18:54:17.070323 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.070305    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"
Sep 11 18:54:17.072767 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.072742    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:54:17.072888 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.072746    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:54:17.080159 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.080135    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"]
Sep 11 18:54:17.164379 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.164346    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5781ff24-910c-451c-b41f-708a28101ee1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9\" (UID: \"5781ff24-910c-451c-b41f-708a28101ee1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"
Sep 11 18:54:17.265752 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.265718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5781ff24-910c-451c-b41f-708a28101ee1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9\" (UID: \"5781ff24-910c-451c-b41f-708a28101ee1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"
Sep 11 18:54:17.266323 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.266303    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5781ff24-910c-451c-b41f-708a28101ee1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9\" (UID: \"5781ff24-910c-451c-b41f-708a28101ee1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"
Sep 11 18:54:17.381134 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.381045    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"
Sep 11 18:54:17.650815 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.650787    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"]
Sep 11 18:54:17.652715 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:54:17.652663    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5781ff24_910c_451c_b41f_708a28101ee1.slice/crio-c90c14eccaeebdaaf337c518783cb6642413b8fb16e97cf8794153713dc1796c WatchSource:0}: Error finding container c90c14eccaeebdaaf337c518783cb6642413b8fb16e97cf8794153713dc1796c: Status 404 returned error can't find the container with id c90c14eccaeebdaaf337c518783cb6642413b8fb16e97cf8794153713dc1796c
Sep 11 18:54:17.835152 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:17.835118    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9" event={"ID":"5781ff24-910c-451c-b41f-708a28101ee1","Type":"ContainerStarted","Data":"c90c14eccaeebdaaf337c518783cb6642413b8fb16e97cf8794153713dc1796c"}
Sep 11 18:54:18.589558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:18.589479    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:54:18.589558 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:18.589533    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:54:18.842858 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:18.842769    2578 generic.go:356] "Generic (PLEG): container finished" podID="5781ff24-910c-451c-b41f-708a28101ee1" containerID="7801f968a5be007903c4705abef9cfe261b39b6b82bf3cfcc578a4199bd2ca1f" exitCode=0
Sep 11 18:54:18.843232 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:18.842857    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9" event={"ID":"5781ff24-910c-451c-b41f-708a28101ee1","Type":"ContainerDied","Data":"7801f968a5be007903c4705abef9cfe261b39b6b82bf3cfcc578a4199bd2ca1f"}
Sep 11 18:54:19.980127 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:19.980103    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"
Sep 11 18:54:20.092493 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:20.092446    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5781ff24-910c-451c-b41f-708a28101ee1-maas-api-service-ca\") pod \"5781ff24-910c-451c-b41f-708a28101ee1\" (UID: \"5781ff24-910c-451c-b41f-708a28101ee1\") "
Sep 11 18:54:20.092886 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:20.092862    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5781ff24-910c-451c-b41f-708a28101ee1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5781ff24-910c-451c-b41f-708a28101ee1" (UID: "5781ff24-910c-451c-b41f-708a28101ee1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:54:20.193274 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:20.193225    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5781ff24-910c-451c-b41f-708a28101ee1-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:20.854797 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:20.854763    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"
Sep 11 18:54:20.854975 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:20.854803    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9" event={"ID":"5781ff24-910c-451c-b41f-708a28101ee1","Type":"ContainerDied","Data":"c90c14eccaeebdaaf337c518783cb6642413b8fb16e97cf8794153713dc1796c"}
Sep 11 18:54:20.854975 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:20.854883    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c90c14eccaeebdaaf337c518783cb6642413b8fb16e97cf8794153713dc1796c"
Sep 11 18:54:21.863639 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:21.863609    2578 generic.go:356] "Generic (PLEG): container finished" podID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerID="d6e194a81d6d46225c98e91b27e8cfcebb12bfd7a5d2f77355045cced2d34803" exitCode=0
Sep 11 18:54:21.864100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:21.863664    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" event={"ID":"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc","Type":"ContainerDied","Data":"d6e194a81d6d46225c98e91b27e8cfcebb12bfd7a5d2f77355045cced2d34803"}
Sep 11 18:54:21.864100 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:21.863737    2578 scope.go:117] "RemoveContainer" containerID="bd0cecb54f404f24d4498b90b4c68a6f349ce00959437458a1934e77b65bab56"
Sep 11 18:54:21.929422 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:21.929398    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:54:22.009953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.009874    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-podinfo\") pod \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") "
Sep 11 18:54:22.009953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.009922    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pfw7\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-kube-api-access-7pfw7\") pod \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") "
Sep 11 18:54:22.009953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.009954    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-certs\") pod \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") "
Sep 11 18:54:22.010200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.009997    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-data\") pod \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") "
Sep 11 18:54:22.010200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-token\") pod \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") "
Sep 11 18:54:22.010200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010067    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-socket\") pod \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") "
Sep 11 18:54:22.010200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010090    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-envoy\") pod \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") "
Sep 11 18:54:22.010200 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010195    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istiod-ca-cert\") pod \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") "
Sep 11 18:54:22.010463 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010227    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-credential-socket\") pod \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\" (UID: \"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc\") "
Sep 11 18:54:22.010463 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010299    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" (UID: "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:22.010463 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010425    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" (UID: "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:22.010616 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010580    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" (UID: "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:22.010671 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010642    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:22.010671 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010662    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:22.010671 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010659    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" (UID: "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:22.010852 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.010702    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" (UID: "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:54:22.012279 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.012252    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" (UID: "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:54:22.012391 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.012288    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" (UID: "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:22.012520 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.012505    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-kube-api-access-7pfw7" (OuterVolumeSpecName: "kube-api-access-7pfw7") pod "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" (UID: "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc"). InnerVolumeSpecName "kube-api-access-7pfw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:54:22.012580 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.012568    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" (UID: "d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:54:22.111904 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.111870    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pfw7\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-kube-api-access-7pfw7\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:22.111904 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.111898    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:22.111904 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.111907    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:22.111904 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.111915    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:22.112202 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.111922    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:22.112202 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.111930    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:22.112202 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.111937    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:22.870012 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.869983    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"
Sep 11 18:54:22.870412 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.870036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv" event={"ID":"d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc","Type":"ContainerDied","Data":"76e6a99a4035565e6a6bb15aaf7926fc2401c2670466b2789057605e5c6afebe"}
Sep 11 18:54:22.870412 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.870095    2578 scope.go:117] "RemoveContainer" containerID="d6e194a81d6d46225c98e91b27e8cfcebb12bfd7a5d2f77355045cced2d34803"
Sep 11 18:54:22.893573 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.893548    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"]
Sep 11 18:54:22.897941 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:22.897920    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ed2c8-openshift-default-6fcff4b696-wlfsv"]
Sep 11 18:54:23.410076 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:23.410036    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" path="/var/lib/kubelet/pods/d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc/volumes"
Sep 11 18:54:27.759904 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:27.759868    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n"]
Sep 11 18:54:27.761859 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:27.761836    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d90e71-29819205-fh77n"]
Sep 11 18:54:27.949722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:27.949688    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"]
Sep 11 18:54:27.952326 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:27.952300    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d90e71-76fn9"]
Sep 11 18:54:29.409874 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:29.409843    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5781ff24-910c-451c-b41f-708a28101ee1" path="/var/lib/kubelet/pods/5781ff24-910c-451c-b41f-708a28101ee1/volumes"
Sep 11 18:54:29.410245 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:29.410132    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74" path="/var/lib/kubelet/pods/bf0f4b6b-cb3f-4cbd-a8cb-0c9693b1fd74/volumes"
Sep 11 18:54:34.728883 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:34.728845    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"]
Sep 11 18:54:34.729315 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:34.729084    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy" containerID="cri-o://60c2504a0262ca3fbacf7d348af1eed8a26bf0f19fc04ae21ac589b87e954b83" gracePeriod=30
Sep 11 18:54:35.103168 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.103080    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"]
Sep 11 18:54:35.103442 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.103428    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5781ff24-910c-451c-b41f-708a28101ee1" containerName="revoke-keys"
Sep 11 18:54:35.103491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.103444    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5781ff24-910c-451c-b41f-708a28101ee1" containerName="revoke-keys"
Sep 11 18:54:35.103491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.103462    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy"
Sep 11 18:54:35.103491 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.103468    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy"
Sep 11 18:54:35.103591 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.103533    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy"
Sep 11 18:54:35.103591 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.103541    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy"
Sep 11 18:54:35.103591 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.103548    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5781ff24-910c-451c-b41f-708a28101ee1" containerName="revoke-keys"
Sep 11 18:54:35.107828 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.107807    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"
Sep 11 18:54:35.110261 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.110240    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 18:54:35.110597 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.110576    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:54:35.115916 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.115893    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"]
Sep 11 18:54:35.233580 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.233539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca07bad5-a4bd-49ec-9f95-58df218cf209-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6\" (UID: \"ca07bad5-a4bd-49ec-9f95-58df218cf209\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"
Sep 11 18:54:35.334746 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.334670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca07bad5-a4bd-49ec-9f95-58df218cf209-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6\" (UID: \"ca07bad5-a4bd-49ec-9f95-58df218cf209\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"
Sep 11 18:54:35.335304 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.335282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca07bad5-a4bd-49ec-9f95-58df218cf209-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6\" (UID: \"ca07bad5-a4bd-49ec-9f95-58df218cf209\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"
Sep 11 18:54:35.418632 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.418607    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"
Sep 11 18:54:35.694788 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.694762    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"]
Sep 11 18:54:35.697204 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:54:35.697178    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca07bad5_a4bd_49ec_9f95_58df218cf209.slice/crio-2a9468da9cc6515da4f121b6cc11436acbe68488da9d76b7cf521cea469a1bad WatchSource:0}: Error finding container 2a9468da9cc6515da4f121b6cc11436acbe68488da9d76b7cf521cea469a1bad: Status 404 returned error can't find the container with id 2a9468da9cc6515da4f121b6cc11436acbe68488da9d76b7cf521cea469a1bad
Sep 11 18:54:35.939582 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:35.939544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6" event={"ID":"ca07bad5-a4bd-49ec-9f95-58df218cf209","Type":"ContainerStarted","Data":"2a9468da9cc6515da4f121b6cc11436acbe68488da9d76b7cf521cea469a1bad"}
Sep 11 18:54:36.947536 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:36.947504    2578 generic.go:356] "Generic (PLEG): container finished" podID="ca07bad5-a4bd-49ec-9f95-58df218cf209" containerID="894fb693cea34e4c8a9ebe61a664888ecf632807888f78c95e7cb37cacde220d" exitCode=0
Sep 11 18:54:36.947939 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:36.947593    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6" event={"ID":"ca07bad5-a4bd-49ec-9f95-58df218cf209","Type":"ContainerDied","Data":"894fb693cea34e4c8a9ebe61a664888ecf632807888f78c95e7cb37cacde220d"}
Sep 11 18:54:38.020358 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:38.020320    2578 scope.go:117] "RemoveContainer" containerID="b5f62aa1084349647bbb7a63119f7717ff8951eb6c758ddc87d678d37d60a8ff"
Sep 11 18:54:38.094490 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:38.094463    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"
Sep 11 18:54:38.259838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:38.259747    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca07bad5-a4bd-49ec-9f95-58df218cf209-maas-api-service-ca\") pod \"ca07bad5-a4bd-49ec-9f95-58df218cf209\" (UID: \"ca07bad5-a4bd-49ec-9f95-58df218cf209\") "
Sep 11 18:54:38.260125 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:38.260098    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca07bad5-a4bd-49ec-9f95-58df218cf209-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca07bad5-a4bd-49ec-9f95-58df218cf209" (UID: "ca07bad5-a4bd-49ec-9f95-58df218cf209"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:54:38.361460 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:38.361421    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca07bad5-a4bd-49ec-9f95-58df218cf209-maas-api-service-ca\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:38.964742 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:38.964706    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6" event={"ID":"ca07bad5-a4bd-49ec-9f95-58df218cf209","Type":"ContainerDied","Data":"2a9468da9cc6515da4f121b6cc11436acbe68488da9d76b7cf521cea469a1bad"}
Sep 11 18:54:38.964922 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:38.964768    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a9468da9cc6515da4f121b6cc11436acbe68488da9d76b7cf521cea469a1bad"
Sep 11 18:54:38.964922 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:38.964722    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"
Sep 11 18:54:39.974286 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:39.974255    2578 generic.go:356] "Generic (PLEG): container finished" podID="840dc218-df17-4626-b356-b01616169418" containerID="60c2504a0262ca3fbacf7d348af1eed8a26bf0f19fc04ae21ac589b87e954b83" exitCode=0
Sep 11 18:54:39.974714 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:39.974331    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" event={"ID":"840dc218-df17-4626-b356-b01616169418","Type":"ContainerDied","Data":"60c2504a0262ca3fbacf7d348af1eed8a26bf0f19fc04ae21ac589b87e954b83"}
Sep 11 18:54:39.974714 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:39.974390    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c" event={"ID":"840dc218-df17-4626-b356-b01616169418","Type":"ContainerDied","Data":"13fd67fac58bf0c4134529f3338b57f4ed7ad2675eaf67f340ec1133c4fa840a"}
Sep 11 18:54:39.974714 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:39.974408    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13fd67fac58bf0c4134529f3338b57f4ed7ad2675eaf67f340ec1133c4fa840a"
Sep 11 18:54:39.974714 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:39.974427    2578 scope.go:117] "RemoveContainer" containerID="34f8df5ade8cc09f2f478ffe5906b1bebede3c1b902a063ab27cd6d41795d40e"
Sep 11 18:54:39.977933 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:39.977914    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:54:40.077660 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077569    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-envoy\") pod \"840dc218-df17-4626-b356-b01616169418\" (UID: \"840dc218-df17-4626-b356-b01616169418\") "
Sep 11 18:54:40.077660 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077609    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-certs\") pod \"840dc218-df17-4626-b356-b01616169418\" (UID: \"840dc218-df17-4626-b356-b01616169418\") "
Sep 11 18:54:40.077660 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077641    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-socket\") pod \"840dc218-df17-4626-b356-b01616169418\" (UID: \"840dc218-df17-4626-b356-b01616169418\") "
Sep 11 18:54:40.077989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077663    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78wq5\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-kube-api-access-78wq5\") pod \"840dc218-df17-4626-b356-b01616169418\" (UID: \"840dc218-df17-4626-b356-b01616169418\") "
Sep 11 18:54:40.077989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077722    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/840dc218-df17-4626-b356-b01616169418-istio-podinfo\") pod \"840dc218-df17-4626-b356-b01616169418\" (UID: \"840dc218-df17-4626-b356-b01616169418\") "
Sep 11 18:54:40.077989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077764    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-istio-token\") pod \"840dc218-df17-4626-b356-b01616169418\" (UID: \"840dc218-df17-4626-b356-b01616169418\") "
Sep 11 18:54:40.077989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077795    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-data\") pod \"840dc218-df17-4626-b356-b01616169418\" (UID: \"840dc218-df17-4626-b356-b01616169418\") "
Sep 11 18:54:40.077989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077930    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "840dc218-df17-4626-b356-b01616169418" (UID: "840dc218-df17-4626-b356-b01616169418"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:40.077989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077939    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "840dc218-df17-4626-b356-b01616169418" (UID: "840dc218-df17-4626-b356-b01616169418"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:40.077989 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.077974    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-credential-socket\") pod \"840dc218-df17-4626-b356-b01616169418\" (UID: \"840dc218-df17-4626-b356-b01616169418\") "
Sep 11 18:54:40.078354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.078086    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/840dc218-df17-4626-b356-b01616169418-istiod-ca-cert\") pod \"840dc218-df17-4626-b356-b01616169418\" (UID: \"840dc218-df17-4626-b356-b01616169418\") "
Sep 11 18:54:40.078354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.078095    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "840dc218-df17-4626-b356-b01616169418" (UID: "840dc218-df17-4626-b356-b01616169418"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:40.078354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.078112    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-data" (OuterVolumeSpecName: "istio-data") pod "840dc218-df17-4626-b356-b01616169418" (UID: "840dc218-df17-4626-b356-b01616169418"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:40.078522 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.078408    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:40.078522 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.078430    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:40.078522 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.078443    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:40.078522 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.078456    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:40.078522 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.078510    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/840dc218-df17-4626-b356-b01616169418-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "840dc218-df17-4626-b356-b01616169418" (UID: "840dc218-df17-4626-b356-b01616169418"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:54:40.079903 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.079878    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "840dc218-df17-4626-b356-b01616169418" (UID: "840dc218-df17-4626-b356-b01616169418"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:40.080096 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.080074    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-kube-api-access-78wq5" (OuterVolumeSpecName: "kube-api-access-78wq5") pod "840dc218-df17-4626-b356-b01616169418" (UID: "840dc218-df17-4626-b356-b01616169418"). InnerVolumeSpecName "kube-api-access-78wq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:54:40.080143 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.080084    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-istio-token" (OuterVolumeSpecName: "istio-token") pod "840dc218-df17-4626-b356-b01616169418" (UID: "840dc218-df17-4626-b356-b01616169418"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:54:40.080185 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.080140    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/840dc218-df17-4626-b356-b01616169418-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "840dc218-df17-4626-b356-b01616169418" (UID: "840dc218-df17-4626-b356-b01616169418"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:54:40.178953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.178918    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78wq5\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-kube-api-access-78wq5\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:40.178953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.178949    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/840dc218-df17-4626-b356-b01616169418-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:40.178953 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.178959    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/840dc218-df17-4626-b356-b01616169418-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:40.179186 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.178968    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/840dc218-df17-4626-b356-b01616169418-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:40.179186 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.178976    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/840dc218-df17-4626-b356-b01616169418-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:40.982395 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:40.982369    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"
Sep 11 18:54:41.015486 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:41.015455    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"]
Sep 11 18:54:41.022273 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:41.022243    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d90e71-openshift-default-77994bbb96-9wf2c"]
Sep 11 18:54:41.410742 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:41.410711    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="840dc218-df17-4626-b356-b01616169418" path="/var/lib/kubelet/pods/840dc218-df17-4626-b356-b01616169418/volumes"
Sep 11 18:54:45.588906 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:45.588860    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv"]
Sep 11 18:54:45.590578 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:45.590556    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-553671-29819205-crqvv"]
Sep 11 18:54:45.615661 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:45.615627    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"]
Sep 11 18:54:45.616570 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:45.615977    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8" podUID="451892e8-e092-45c3-b2d5-a3353a84efe7" containerName="payload-processing" containerID="cri-o://5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9" gracePeriod=30
Sep 11 18:54:45.768354 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:45.768317    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"]
Sep 11 18:54:45.770750 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:45.770725    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-553671-bg2w6"]
Sep 11 18:54:45.869044 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:45.869014    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:54:46.007825 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.007788    2578 generic.go:356] "Generic (PLEG): container finished" podID="451892e8-e092-45c3-b2d5-a3353a84efe7" containerID="5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9" exitCode=0
Sep 11 18:54:46.008005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.007837    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"
Sep 11 18:54:46.008005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.007847    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8" event={"ID":"451892e8-e092-45c3-b2d5-a3353a84efe7","Type":"ContainerDied","Data":"5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9"}
Sep 11 18:54:46.008005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.007899    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8" event={"ID":"451892e8-e092-45c3-b2d5-a3353a84efe7","Type":"ContainerDied","Data":"075ab674226d958727a5fa7543a94d64cff26efa6a5d2392e8a77c7db7dda2f0"}
Sep 11 18:54:46.008005 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.007930    2578 scope.go:117] "RemoveContainer" containerID="5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9"
Sep 11 18:54:46.016245 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.016227    2578 scope.go:117] "RemoveContainer" containerID="5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9"
Sep 11 18:54:46.016491 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:54:46.016473    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9\": container with ID starting with 5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9 not found: ID does not exist" containerID="5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9"
Sep 11 18:54:46.016567 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.016498    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9"} err="failed to get container status \"5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9\": rpc error: code = NotFound desc = could not find container \"5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9\": container with ID starting with 5e5a2dafb2cd15779401c20682ea18cf5d9265b250fcbb6c40235633a1a951b9 not found: ID does not exist"
Sep 11 18:54:46.032844 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.032823    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr8l6\" (UniqueName: \"kubernetes.io/projected/451892e8-e092-45c3-b2d5-a3353a84efe7-kube-api-access-hr8l6\") pod \"451892e8-e092-45c3-b2d5-a3353a84efe7\" (UID: \"451892e8-e092-45c3-b2d5-a3353a84efe7\") "
Sep 11 18:54:46.032942 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.032930    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/451892e8-e092-45c3-b2d5-a3353a84efe7-plugins-config-volume\") pod \"451892e8-e092-45c3-b2d5-a3353a84efe7\" (UID: \"451892e8-e092-45c3-b2d5-a3353a84efe7\") "
Sep 11 18:54:46.033334 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.033307    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/451892e8-e092-45c3-b2d5-a3353a84efe7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "451892e8-e092-45c3-b2d5-a3353a84efe7" (UID: "451892e8-e092-45c3-b2d5-a3353a84efe7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:54:46.034758 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.034732    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/451892e8-e092-45c3-b2d5-a3353a84efe7-kube-api-access-hr8l6" (OuterVolumeSpecName: "kube-api-access-hr8l6") pod "451892e8-e092-45c3-b2d5-a3353a84efe7" (UID: "451892e8-e092-45c3-b2d5-a3353a84efe7"). InnerVolumeSpecName "kube-api-access-hr8l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:54:46.134530 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.134448    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr8l6\" (UniqueName: \"kubernetes.io/projected/451892e8-e092-45c3-b2d5-a3353a84efe7-kube-api-access-hr8l6\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:46.134530 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.134481    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/451892e8-e092-45c3-b2d5-a3353a84efe7-plugins-config-volume\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:46.329116 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.329085    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"]
Sep 11 18:54:46.333043 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:46.333020    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-553671-685fb49587-6srz8"]
Sep 11 18:54:47.415037 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:47.415005    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="451892e8-e092-45c3-b2d5-a3353a84efe7" path="/var/lib/kubelet/pods/451892e8-e092-45c3-b2d5-a3353a84efe7/volumes"
Sep 11 18:54:47.415423 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:47.415274    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98fb8bec-70de-40b6-b9ba-48564b3d0773" path="/var/lib/kubelet/pods/98fb8bec-70de-40b6-b9ba-48564b3d0773/volumes"
Sep 11 18:54:47.415526 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:47.415513    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca07bad5-a4bd-49ec-9f95-58df218cf209" path="/var/lib/kubelet/pods/ca07bad5-a4bd-49ec-9f95-58df218cf209/volumes"
Sep 11 18:54:52.260478 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:52.260437    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"]
Sep 11 18:54:52.261007 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:52.260977    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" containerID="cri-o://0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06" gracePeriod=30
Sep 11 18:54:55.966763 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:55.966723    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:54:55.967141 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:55.966804    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:54:57.518473 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.518445    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:54:57.632722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.632599    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istiod-ca-cert\") pod \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") "
Sep 11 18:54:57.632722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.632639    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhgdw\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-kube-api-access-jhgdw\") pod \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") "
Sep 11 18:54:57.632722 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.632660    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-token\") pod \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") "
Sep 11 18:54:57.633026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.632740    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-podinfo\") pod \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") "
Sep 11 18:54:57.633026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.632785    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-credential-socket\") pod \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") "
Sep 11 18:54:57.633026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.632800    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-data\") pod \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") "
Sep 11 18:54:57.633026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.632814    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-certs\") pod \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") "
Sep 11 18:54:57.633026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.632848    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-socket\") pod \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") "
Sep 11 18:54:57.633026 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.632900    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-envoy\") pod \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\" (UID: \"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6\") "
Sep 11 18:54:57.633335 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.633072    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" (UID: "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:54:57.633472 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.633431    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-data" (OuterVolumeSpecName: "istio-data") pod "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" (UID: "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:57.633613 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.633466    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" (UID: "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:57.633613 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.633482    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" (UID: "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:57.633613 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.633536    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" (UID: "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:57.635101 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.635072    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" (UID: "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:54:57.635276 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.635259    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-token" (OuterVolumeSpecName: "istio-token") pod "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" (UID: "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:54:57.635434 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.635412    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" (UID: "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:54:57.635524 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.635431    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-kube-api-access-jhgdw" (OuterVolumeSpecName: "kube-api-access-jhgdw") pod "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" (UID: "acddc5e8-23b0-4f9e-be54-28f3d2a09ca6"). InnerVolumeSpecName "kube-api-access-jhgdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:54:57.733897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.733860    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:57.733897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.733888    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-envoy\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:57.733897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.733897    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istiod-ca-cert\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:57.733897 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.733906    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhgdw\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-kube-api-access-jhgdw\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:57.734174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.733915    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-token\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:57.734174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.733923    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-podinfo\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:57.734174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.733930    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-credential-socket\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:57.734174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.733938    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-istio-data\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:57.734174 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:57.733946    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6-workload-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 18:54:58.077661 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.077629    2578 generic.go:356] "Generic (PLEG): container finished" podID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerID="0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06" exitCode=0
Sep 11 18:54:58.077876 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.077705    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"
Sep 11 18:54:58.077876 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.077715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" event={"ID":"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6","Type":"ContainerDied","Data":"0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06"}
Sep 11 18:54:58.077876 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.077775    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq" event={"ID":"acddc5e8-23b0-4f9e-be54-28f3d2a09ca6","Type":"ContainerDied","Data":"aab04e9adf9d83c4cd6121904a503e4d5e964764a48be34744cfc4f7beaa5d37"}
Sep 11 18:54:58.077876 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.077802    2578 scope.go:117] "RemoveContainer" containerID="0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06"
Sep 11 18:54:58.086746 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.086729    2578 scope.go:117] "RemoveContainer" containerID="ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf"
Sep 11 18:54:58.095103 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.095084    2578 scope.go:117] "RemoveContainer" containerID="0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06"
Sep 11 18:54:58.095390 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:54:58.095372    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06\": container with ID starting with 0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06 not found: ID does not exist" containerID="0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06"
Sep 11 18:54:58.095434 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.095402    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06"} err="failed to get container status \"0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06\": rpc error: code = NotFound desc = could not find container \"0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06\": container with ID starting with 0d04a67614a84ac03532a2aad40f461c29714720ac07a82e8ae509c277ab2b06 not found: ID does not exist"
Sep 11 18:54:58.095434 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.095418    2578 scope.go:117] "RemoveContainer" containerID="ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf"
Sep 11 18:54:58.095695 ip-10-0-141-82 kubenswrapper[2578]: E0911 18:54:58.095660    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf\": container with ID starting with ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf not found: ID does not exist" containerID="ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf"
Sep 11 18:54:58.095739 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.095705    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf"} err="failed to get container status \"ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf\": rpc error: code = NotFound desc = could not find container \"ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf\": container with ID starting with ff1d62c16476eaddbef05eed5f340e7d7c2450220fa9ae487c8b3e349154afaf not found: ID does not exist"
Sep 11 18:54:58.101588 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.101567    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"]
Sep 11 18:54:58.107090 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:58.107069    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-553671-openshift-default-84cbc99bdd-wkjpq"]
Sep 11 18:54:59.409524 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:54:59.409485    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" path="/var/lib/kubelet/pods/acddc5e8-23b0-4f9e-be54-28f3d2a09ca6/volumes"
Sep 11 18:55:37.591896 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:55:37.591866    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:55:37.595795 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:55:37.595770    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:55:37.603401 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:55:37.603383    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 18:55:37.607211 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:55:37.607190    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 18:55:38.054274 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:55:38.054238    2578 scope.go:117] "RemoveContainer" containerID="1e4290d903f8416d14a9ef6020d0580f1e5250416b3e8e9ecfa023b1db02955b"
Sep 11 18:55:38.063753 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:55:38.063691    2578 scope.go:117] "RemoveContainer" containerID="60c2504a0262ca3fbacf7d348af1eed8a26bf0f19fc04ae21ac589b87e954b83"
Sep 11 18:58:38.107303 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:58:38.107269    2578 scope.go:117] "RemoveContainer" containerID="56209aee40ad4511bfcc01af5d65edc86f9c366c0b4657418b7fe380522afafa"
Sep 11 18:59:20.032290 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032257    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"]
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032596    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy"
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032610    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy"
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032619    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca07bad5-a4bd-49ec-9f95-58df218cf209" containerName="revoke-keys"
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032627    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca07bad5-a4bd-49ec-9f95-58df218cf209" containerName="revoke-keys"
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032641    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy"
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032646    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a2c3a4-7346-402e-bb2e-f5acbf7fd8fc" containerName="istio-proxy"
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032661    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy"
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032669    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy"
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032700    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy"
Sep 11 18:59:20.032716 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032709    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy"
Sep 11 18:59:20.033083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032723    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="451892e8-e092-45c3-b2d5-a3353a84efe7" containerName="payload-processing"
Sep 11 18:59:20.033083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032732    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="451892e8-e092-45c3-b2d5-a3353a84efe7" containerName="payload-processing"
Sep 11 18:59:20.033083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032797    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy"
Sep 11 18:59:20.033083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032804    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy"
Sep 11 18:59:20.033083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032813    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy"
Sep 11 18:59:20.033083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032823    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca07bad5-a4bd-49ec-9f95-58df218cf209" containerName="revoke-keys"
Sep 11 18:59:20.033083 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.032834    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="451892e8-e092-45c3-b2d5-a3353a84efe7" containerName="payload-processing"
Sep 11 18:59:20.035626 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.035606    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.037847 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.037826    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-c6d92ea4-kserve-self-signed-certs\""
Sep 11 18:59:20.047774 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.047753    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"]
Sep 11 18:59:20.101356 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.101324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-tmp-dir\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.101356 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.101357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bef667-7e0e-4c90-9cba-d0fe35652d31-tls-certs\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.101529 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.101385    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-home\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.101529 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.101439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-model-cache\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.101529 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.101489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxxv2\" (UniqueName: \"kubernetes.io/projected/37bef667-7e0e-4c90-9cba-d0fe35652d31-kube-api-access-qxxv2\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.101529 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.101520    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-dshm\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.202857 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.202824    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxv2\" (UniqueName: \"kubernetes.io/projected/37bef667-7e0e-4c90-9cba-d0fe35652d31-kube-api-access-qxxv2\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.203043 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.202871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-dshm\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.203043 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.202915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-tmp-dir\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.203043 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.202941    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bef667-7e0e-4c90-9cba-d0fe35652d31-tls-certs\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.203043 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.202969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-home\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.203043 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.203025    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-model-cache\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.203418 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.203393    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-tmp-dir\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.203495 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.203425    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-home\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.203495 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.203432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-model-cache\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.205282 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.205255    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-dshm\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.205449 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.205430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bef667-7e0e-4c90-9cba-d0fe35652d31-tls-certs\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.210762 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.210745    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxxv2\" (UniqueName: \"kubernetes.io/projected/37bef667-7e0e-4c90-9cba-d0fe35652d31-kube-api-access-qxxv2\") pod \"e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") " pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.348158 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.348071    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:20.621112 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:20.621039    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"]
Sep 11 18:59:20.624433 ip-10-0-141-82 kubenswrapper[2578]: W0911 18:59:20.624403    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37bef667_7e0e_4c90_9cba_d0fe35652d31.slice/crio-10a1fba62fee4f2bb8faf0e802c811c2777220d233632c10a3c30569c3575e8a WatchSource:0}: Error finding container 10a1fba62fee4f2bb8faf0e802c811c2777220d233632c10a3c30569c3575e8a: Status 404 returned error can't find the container with id 10a1fba62fee4f2bb8faf0e802c811c2777220d233632c10a3c30569c3575e8a
Sep 11 18:59:21.358838 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:21.358801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz" event={"ID":"37bef667-7e0e-4c90-9cba-d0fe35652d31","Type":"ContainerStarted","Data":"9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d"}
Sep 11 18:59:21.359269 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:21.358850    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz" event={"ID":"37bef667-7e0e-4c90-9cba-d0fe35652d31","Type":"ContainerStarted","Data":"10a1fba62fee4f2bb8faf0e802c811c2777220d233632c10a3c30569c3575e8a"}
Sep 11 18:59:21.377341 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:21.377273    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz" podStartSLOduration=1.377256961 podStartE2EDuration="1.377256961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:59:21.376008387 +0000 UTC m=+2624.411225058" watchObservedRunningTime="2026-09-11 18:59:21.377256961 +0000 UTC m=+2624.412473630"
Sep 11 18:59:30.349214 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:30.349178    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:30.349725 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:30.349276    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:30.361845 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:30.361821    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:30.416607 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:30.416579    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 18:59:38.128843 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:38.128815    2578 scope.go:117] "RemoveContainer" containerID="f4260afc919fce29a0d53ae62bd43072d75bff1f47b9ee224a3ac43eda94b64f"
Sep 11 18:59:54.674741 ip-10-0-141-82 kubenswrapper[2578]: I0911 18:59:54.674640    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"]
Sep 11 19:00:00.131073 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.131039    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv"]
Sep 11 19:00:00.131449 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.131370    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy"
Sep 11 19:00:00.131449 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.131380    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="acddc5e8-23b0-4f9e-be54-28f3d2a09ca6" containerName="istio-proxy"
Sep 11 19:00:00.131524 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.131457    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="840dc218-df17-4626-b356-b01616169418" containerName="istio-proxy"
Sep 11 19:00:00.133905 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.133887    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv"
Sep 11 19:00:00.136320 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.136303    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zpmfp\""
Sep 11 19:00:00.142839 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.142805    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv"]
Sep 11 19:00:00.144821 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.144802    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv"
Sep 11 19:00:00.417204 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.417178    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv"]
Sep 11 19:00:00.419328 ip-10-0-141-82 kubenswrapper[2578]: W0911 19:00:00.419290    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b04d663_e4e6_4f1d_8cc0_7f5ff4007553.slice/crio-355ba7aec3b4eb6b86632bfc3d5a35e21518d5f278f7dcae52febd07eb2db219 WatchSource:0}: Error finding container 355ba7aec3b4eb6b86632bfc3d5a35e21518d5f278f7dcae52febd07eb2db219: Status 404 returned error can't find the container with id 355ba7aec3b4eb6b86632bfc3d5a35e21518d5f278f7dcae52febd07eb2db219
Sep 11 19:00:00.550648 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:00.550615    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv" event={"ID":"6b04d663-e4e6-4f1d-8cc0-7f5ff4007553","Type":"ContainerStarted","Data":"355ba7aec3b4eb6b86632bfc3d5a35e21518d5f278f7dcae52febd07eb2db219"}
Sep 11 19:00:01.556776 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:01.556662    2578 generic.go:356] "Generic (PLEG): container finished" podID="6b04d663-e4e6-4f1d-8cc0-7f5ff4007553" containerID="328e40022f01a3a8e7e71252264e23dc628803dff316c77cbb9408ca2cc2eb38" exitCode=0
Sep 11 19:00:01.556776 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:01.556746    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv" event={"ID":"6b04d663-e4e6-4f1d-8cc0-7f5ff4007553","Type":"ContainerDied","Data":"328e40022f01a3a8e7e71252264e23dc628803dff316c77cbb9408ca2cc2eb38"}
Sep 11 19:00:02.713325 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:02.713298    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv"
Sep 11 19:00:03.568205 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:03.568174    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv"
Sep 11 19:00:03.568376 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:03.568177    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819220-zvwmv" event={"ID":"6b04d663-e4e6-4f1d-8cc0-7f5ff4007553","Type":"ContainerDied","Data":"355ba7aec3b4eb6b86632bfc3d5a35e21518d5f278f7dcae52febd07eb2db219"}
Sep 11 19:00:03.568376 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:03.568299    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="355ba7aec3b4eb6b86632bfc3d5a35e21518d5f278f7dcae52febd07eb2db219"
Sep 11 19:00:09.694304 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:09.694251    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz" podUID="37bef667-7e0e-4c90-9cba-d0fe35652d31" containerName="main" containerID="cri-o://9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d" gracePeriod=45
Sep 11 19:00:09.968847 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:09.968819    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 19:00:10.055751 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.055718    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-home\") pod \"37bef667-7e0e-4c90-9cba-d0fe35652d31\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") "
Sep 11 19:00:10.055910 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.055780    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-tmp-dir\") pod \"37bef667-7e0e-4c90-9cba-d0fe35652d31\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") "
Sep 11 19:00:10.055910 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.055812    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-model-cache\") pod \"37bef667-7e0e-4c90-9cba-d0fe35652d31\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") "
Sep 11 19:00:10.055910 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.055840    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxxv2\" (UniqueName: \"kubernetes.io/projected/37bef667-7e0e-4c90-9cba-d0fe35652d31-kube-api-access-qxxv2\") pod \"37bef667-7e0e-4c90-9cba-d0fe35652d31\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") "
Sep 11 19:00:10.055910 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.055867    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-dshm\") pod \"37bef667-7e0e-4c90-9cba-d0fe35652d31\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") "
Sep 11 19:00:10.055910 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.055896    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bef667-7e0e-4c90-9cba-d0fe35652d31-tls-certs\") pod \"37bef667-7e0e-4c90-9cba-d0fe35652d31\" (UID: \"37bef667-7e0e-4c90-9cba-d0fe35652d31\") "
Sep 11 19:00:10.056215 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.055987    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-home" (OuterVolumeSpecName: "home") pod "37bef667-7e0e-4c90-9cba-d0fe35652d31" (UID: "37bef667-7e0e-4c90-9cba-d0fe35652d31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:00:10.056215 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.056117    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-model-cache" (OuterVolumeSpecName: "model-cache") pod "37bef667-7e0e-4c90-9cba-d0fe35652d31" (UID: "37bef667-7e0e-4c90-9cba-d0fe35652d31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:00:10.056215 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.056128    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "37bef667-7e0e-4c90-9cba-d0fe35652d31" (UID: "37bef667-7e0e-4c90-9cba-d0fe35652d31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:00:10.056215 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.056184    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-home\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 19:00:10.058127 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.058106    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-dshm" (OuterVolumeSpecName: "dshm") pod "37bef667-7e0e-4c90-9cba-d0fe35652d31" (UID: "37bef667-7e0e-4c90-9cba-d0fe35652d31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 19:00:10.058213 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.058136    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37bef667-7e0e-4c90-9cba-d0fe35652d31-kube-api-access-qxxv2" (OuterVolumeSpecName: "kube-api-access-qxxv2") pod "37bef667-7e0e-4c90-9cba-d0fe35652d31" (UID: "37bef667-7e0e-4c90-9cba-d0fe35652d31"). InnerVolumeSpecName "kube-api-access-qxxv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 19:00:10.058267 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.058206    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37bef667-7e0e-4c90-9cba-d0fe35652d31-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37bef667-7e0e-4c90-9cba-d0fe35652d31" (UID: "37bef667-7e0e-4c90-9cba-d0fe35652d31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 19:00:10.156978 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.156933    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-tmp-dir\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 19:00:10.156978 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.156969    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-model-cache\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 19:00:10.156978 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.156983    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxxv2\" (UniqueName: \"kubernetes.io/projected/37bef667-7e0e-4c90-9cba-d0fe35652d31-kube-api-access-qxxv2\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 19:00:10.157216 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.156995    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bef667-7e0e-4c90-9cba-d0fe35652d31-dshm\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 19:00:10.157216 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.157007    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bef667-7e0e-4c90-9cba-d0fe35652d31-tls-certs\") on node \"ip-10-0-141-82.ec2.internal\" DevicePath \"\""
Sep 11 19:00:10.605673 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.605637    2578 generic.go:356] "Generic (PLEG): container finished" podID="37bef667-7e0e-4c90-9cba-d0fe35652d31" containerID="9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d" exitCode=0
Sep 11 19:00:10.605858 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.605702    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"
Sep 11 19:00:10.605858 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.605729    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz" event={"ID":"37bef667-7e0e-4c90-9cba-d0fe35652d31","Type":"ContainerDied","Data":"9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d"}
Sep 11 19:00:10.605858 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.605782    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz" event={"ID":"37bef667-7e0e-4c90-9cba-d0fe35652d31","Type":"ContainerDied","Data":"10a1fba62fee4f2bb8faf0e802c811c2777220d233632c10a3c30569c3575e8a"}
Sep 11 19:00:10.605858 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.605806    2578 scope.go:117] "RemoveContainer" containerID="9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d"
Sep 11 19:00:10.615263 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.615244    2578 scope.go:117] "RemoveContainer" containerID="9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d"
Sep 11 19:00:10.615524 ip-10-0-141-82 kubenswrapper[2578]: E0911 19:00:10.615509    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d\": container with ID starting with 9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d not found: ID does not exist" containerID="9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d"
Sep 11 19:00:10.615569 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.615532    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d"} err="failed to get container status \"9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d\": rpc error: code = NotFound desc = could not find container \"9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d\": container with ID starting with 9945a6ba8edb08d86fc41c1e7e6c818304565edc00e63ba8e95c456637493d5d not found: ID does not exist"
Sep 11 19:00:10.627749 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.627728    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"]
Sep 11 19:00:10.629932 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:10.629911    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-c6d92ea4-kserve-656848c959-qc7xz"]
Sep 11 19:00:11.409728 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:11.409699    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37bef667-7e0e-4c90-9cba-d0fe35652d31" path="/var/lib/kubelet/pods/37bef667-7e0e-4c90-9cba-d0fe35652d31/volumes"
Sep 11 19:00:37.627192 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:37.627122    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 19:00:37.631467 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:37.631443    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 19:00:37.640188 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:37.640167    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 19:00:37.644187 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:37.644168    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hm89p_4e91f825-89cb-4019-b341-699c8c5a902d/ovn-acl-logging/0.log"
Sep 11 19:00:38.148120 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:38.148091    2578 scope.go:117] "RemoveContainer" containerID="894fb693cea34e4c8a9ebe61a664888ecf632807888f78c95e7cb37cacde220d"
Sep 11 19:00:38.155875 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:38.155850    2578 scope.go:117] "RemoveContainer" containerID="7801f968a5be007903c4705abef9cfe261b39b6b82bf3cfcc578a4199bd2ca1f"
Sep 11 19:00:38.163056 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:38.163029    2578 scope.go:117] "RemoveContainer" containerID="607e59905ba0cfff5be2a2252dfc782bd164c50efa5e11e4dc8c909dfe056b06"
Sep 11 19:00:56.076849 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:56.076812    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g8gpq_04d7afdf-f4a3-42b8-87e5-0612ac2c48b9/manager/0.log"
Sep 11 19:00:56.302760 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:56.302732    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68b7b6b756-c9p8m_ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b/copy-manifests/0.log"
Sep 11 19:00:56.309850 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:56.309827    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68b7b6b756-c9p8m_ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b/manager/0.log"
Sep 11 19:00:57.031407 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:57.031374    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819190-s5wpn_1e613513-91d3-4248-81a4-fc6b038b1339/cleanup/0.log"
Sep 11 19:00:57.137647 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:57.137621    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819205-7vtwt_a9384102-acb4-4ab5-b609-6c8b06daeee6/cleanup/0.log"
Sep 11 19:00:57.249114 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:57.249081    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819220-zvwmv_6b04d663-e4e6-4f1d-8cc0-7f5ff4007553/cleanup/0.log"
Sep 11 19:00:59.159120 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:59.159091    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk_88e240e9-074e-483f-a292-0b1dc581efb9/istio-proxy/0.log"
Sep 11 19:00:59.274969 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:59.274934    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-vpmf6_b911bd70-ea94-468f-92b2-4bcf6b156d31/discovery/0.log"
Sep 11 19:00:59.634723 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:59.634610    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dwbw9_e701e4f0-fa92-4cd0-ae56-737249752574/istio-proxy/1.log"
Sep 11 19:00:59.882705 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:59.882663    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lnhlh_7ddd784b-815b-42c0-bf68-667587dda11b/payload-processing/0.log"
Sep 11 19:00:59.995943 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:00:59.995907    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bf46bd65d-jtnbl_f1d12737-e1eb-4321-9473-e06391ac2642/router/0.log"
Sep 11 19:01:00.937188 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:00.937162    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-dmqhg_7d4bccbe-b752-4345-9e63-7db534a07a69/main/0.log"
Sep 11 19:01:06.196305 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.196269    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5r55n/must-gather-m57br"]
Sep 11 19:01:06.196846 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.196667    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37bef667-7e0e-4c90-9cba-d0fe35652d31" containerName="main"
Sep 11 19:01:06.196846 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.196694    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="37bef667-7e0e-4c90-9cba-d0fe35652d31" containerName="main"
Sep 11 19:01:06.196846 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.196708    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b04d663-e4e6-4f1d-8cc0-7f5ff4007553" containerName="cleanup"
Sep 11 19:01:06.196846 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.196714    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b04d663-e4e6-4f1d-8cc0-7f5ff4007553" containerName="cleanup"
Sep 11 19:01:06.196846 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.196775    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="37bef667-7e0e-4c90-9cba-d0fe35652d31" containerName="main"
Sep 11 19:01:06.196846 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.196786    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b04d663-e4e6-4f1d-8cc0-7f5ff4007553" containerName="cleanup"
Sep 11 19:01:06.199939 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.199921    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5r55n/must-gather-m57br"
Sep 11 19:01:06.202871 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.202836    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5r55n\"/\"openshift-service-ca.crt\""
Sep 11 19:01:06.203032 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.202913    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5r55n\"/\"kube-root-ca.crt\""
Sep 11 19:01:06.203826 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.203804    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5r55n\"/\"default-dockercfg-rblx9\""
Sep 11 19:01:06.274102 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.274061    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5r55n/must-gather-m57br"]
Sep 11 19:01:06.356057 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.356018    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b6473ee0-cfa2-41a3-8a79-4025c8006b55-must-gather-output\") pod \"must-gather-m57br\" (UID: \"b6473ee0-cfa2-41a3-8a79-4025c8006b55\") " pod="openshift-must-gather-5r55n/must-gather-m57br"
Sep 11 19:01:06.356232 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.356072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqlxg\" (UniqueName: \"kubernetes.io/projected/b6473ee0-cfa2-41a3-8a79-4025c8006b55-kube-api-access-hqlxg\") pod \"must-gather-m57br\" (UID: \"b6473ee0-cfa2-41a3-8a79-4025c8006b55\") " pod="openshift-must-gather-5r55n/must-gather-m57br"
Sep 11 19:01:06.457503 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.457410    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b6473ee0-cfa2-41a3-8a79-4025c8006b55-must-gather-output\") pod \"must-gather-m57br\" (UID: \"b6473ee0-cfa2-41a3-8a79-4025c8006b55\") " pod="openshift-must-gather-5r55n/must-gather-m57br"
Sep 11 19:01:06.457503 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.457468    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqlxg\" (UniqueName: \"kubernetes.io/projected/b6473ee0-cfa2-41a3-8a79-4025c8006b55-kube-api-access-hqlxg\") pod \"must-gather-m57br\" (UID: \"b6473ee0-cfa2-41a3-8a79-4025c8006b55\") " pod="openshift-must-gather-5r55n/must-gather-m57br"
Sep 11 19:01:06.457846 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.457825    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b6473ee0-cfa2-41a3-8a79-4025c8006b55-must-gather-output\") pod \"must-gather-m57br\" (UID: \"b6473ee0-cfa2-41a3-8a79-4025c8006b55\") " pod="openshift-must-gather-5r55n/must-gather-m57br"
Sep 11 19:01:06.473469 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.473433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqlxg\" (UniqueName: \"kubernetes.io/projected/b6473ee0-cfa2-41a3-8a79-4025c8006b55-kube-api-access-hqlxg\") pod \"must-gather-m57br\" (UID: \"b6473ee0-cfa2-41a3-8a79-4025c8006b55\") " pod="openshift-must-gather-5r55n/must-gather-m57br"
Sep 11 19:01:06.509532 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.509493    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5r55n/must-gather-m57br"
Sep 11 19:01:06.787432 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.787272    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5r55n/must-gather-m57br"]
Sep 11 19:01:06.790163 ip-10-0-141-82 kubenswrapper[2578]: W0911 19:01:06.790126    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6473ee0_cfa2_41a3_8a79_4025c8006b55.slice/crio-1d7e688479ff1d40a5508e71deba1ade8fb9380982866189cbd217e551b70e00 WatchSource:0}: Error finding container 1d7e688479ff1d40a5508e71deba1ade8fb9380982866189cbd217e551b70e00: Status 404 returned error can't find the container with id 1d7e688479ff1d40a5508e71deba1ade8fb9380982866189cbd217e551b70e00
Sep 11 19:01:06.792118 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.792097    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 19:01:06.872751 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:06.872715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5r55n/must-gather-m57br" event={"ID":"b6473ee0-cfa2-41a3-8a79-4025c8006b55","Type":"ContainerStarted","Data":"1d7e688479ff1d40a5508e71deba1ade8fb9380982866189cbd217e551b70e00"}
Sep 11 19:01:08.885421 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:08.885380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5r55n/must-gather-m57br" event={"ID":"b6473ee0-cfa2-41a3-8a79-4025c8006b55","Type":"ContainerStarted","Data":"9808f2e637abccef900a0a9185a08624b507827de297fac1795f45541689b00b"}
Sep 11 19:01:08.885967 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:08.885435    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5r55n/must-gather-m57br" event={"ID":"b6473ee0-cfa2-41a3-8a79-4025c8006b55","Type":"ContainerStarted","Data":"bfe0314b3da90246bebd2189c76f7fc16bdae0314d30713392e2c76ca9f47164"}
Sep 11 19:01:08.907979 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:08.907905    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5r55n/must-gather-m57br" podStartSLOduration=1.51276538 podStartE2EDuration="2.90787957s" totalImagesPullingTime="1.39511419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 19:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:01:08.903932313 +0000 UTC m=+2731.939148983" watchObservedRunningTime="2026-09-11 19:01:08.90787957 +0000 UTC m=+2731.943096240"
Sep 11 19:01:10.141483 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:10.141447    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wkkbd_6a6009bd-7d00-437f-9cf4-9161d79ab722/global-pull-secret-syncer/0.log"
Sep 11 19:01:10.243642 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:10.243614    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h5km6_ffcd016f-d01c-4d6e-8bca-df7a61e26c91/konnectivity-agent/0.log"
Sep 11 19:01:10.337530 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:10.337501    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-82.ec2.internal_e205fce0f56018a4d3daccefe7a004e7/haproxy/0.log"
Sep 11 19:01:16.646651 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:16.646618    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-8crdq_37e3a74c-b152-456d-a962-3785e8f5ed6d/kube-state-metrics/0.log"
Sep 11 19:01:16.665949 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:16.665912    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-8crdq_37e3a74c-b152-456d-a962-3785e8f5ed6d/kube-rbac-proxy-main/0.log"
Sep 11 19:01:16.693393 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:16.693347    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-8crdq_37e3a74c-b152-456d-a962-3785e8f5ed6d/kube-rbac-proxy-self/0.log"
Sep 11 19:01:16.755450 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:16.755409    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-bxnnd_67e25494-876a-44de-bc40-8feaabc007c2/monitoring-plugin/0.log"
Sep 11 19:01:16.792154 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:16.792116    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbdpb_a69938cd-b755-49e7-85a1-e3775324d382/node-exporter/0.log"
Sep 11 19:01:16.817209 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:16.817165    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbdpb_a69938cd-b755-49e7-85a1-e3775324d382/kube-rbac-proxy/0.log"
Sep 11 19:01:16.842806 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:16.842774    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbdpb_a69938cd-b755-49e7-85a1-e3775324d382/init-textfile/0.log"
Sep 11 19:01:17.105096 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:17.105041    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-gsjvb_7089dec3-ca65-4924-9104-e5029434d860/kube-rbac-proxy-main/0.log"
Sep 11 19:01:17.147533 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:17.147490    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-gsjvb_7089dec3-ca65-4924-9104-e5029434d860/kube-rbac-proxy-self/0.log"
Sep 11 19:01:17.194205 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:17.194172    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-gsjvb_7089dec3-ca65-4924-9104-e5029434d860/openshift-state-metrics/0.log"
Sep 11 19:01:20.008190 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.008132    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/1.log"
Sep 11 19:01:20.016767 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.016730    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-gxksc_c1b338a0-0878-41c0-9bf6-13d543b94c2e/console-operator/2.log"
Sep 11 19:01:20.131559 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.131512    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"]
Sep 11 19:01:20.137906 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.137870    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.143924 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.143889    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"]
Sep 11 19:01:20.200444 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.200403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-sys\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.200444 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.200458    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-lib-modules\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.200714 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.200559    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-proc\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.200714 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.200600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-podres\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.200807 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.200715    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmr7q\" (UniqueName: \"kubernetes.io/projected/9d35d649-6c50-4def-bce6-b3abe38c6174-kube-api-access-wmr7q\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.301767 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.301650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmr7q\" (UniqueName: \"kubernetes.io/projected/9d35d649-6c50-4def-bce6-b3abe38c6174-kube-api-access-wmr7q\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.302362 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.302078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-sys\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.302362 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.302124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-lib-modules\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.302362 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.302178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-proc\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.302362 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.302206    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-podres\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.302362 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.302202    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-sys\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.302362 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.302317    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-lib-modules\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.302362 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.302339    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-proc\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.302845 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.302403    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d35d649-6c50-4def-bce6-b3abe38c6174-podres\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.321128 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.321095    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmr7q\" (UniqueName: \"kubernetes.io/projected/9d35d649-6c50-4def-bce6-b3abe38c6174-kube-api-access-wmr7q\") pod \"perf-node-gather-daemonset-dtcjh\" (UID: \"9d35d649-6c50-4def-bce6-b3abe38c6174\") " pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.456162 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.456125    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.754282 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.754239    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"]
Sep 11 19:01:20.758578 ip-10-0-141-82 kubenswrapper[2578]: W0911 19:01:20.758544    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9d35d649_6c50_4def_bce6_b3abe38c6174.slice/crio-54c64a7e060484ca566a2c2fb614213ff9007ec4eda626c6f5fa144654deea2d WatchSource:0}: Error finding container 54c64a7e060484ca566a2c2fb614213ff9007ec4eda626c6f5fa144654deea2d: Status 404 returned error can't find the container with id 54c64a7e060484ca566a2c2fb614213ff9007ec4eda626c6f5fa144654deea2d
Sep 11 19:01:20.972083 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.972026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh" event={"ID":"9d35d649-6c50-4def-bce6-b3abe38c6174","Type":"ContainerStarted","Data":"721639f0337122d05015847c03651cac492ee666acf9386d6721fe94f18ddc1b"}
Sep 11 19:01:20.972287 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.972166    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh" event={"ID":"9d35d649-6c50-4def-bce6-b3abe38c6174","Type":"ContainerStarted","Data":"54c64a7e060484ca566a2c2fb614213ff9007ec4eda626c6f5fa144654deea2d"}
Sep 11 19:01:20.973353 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.973326    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:20.991308 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:20.991254    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh" podStartSLOduration=0.991237389 podStartE2EDuration="991.237389ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 19:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 19:01:20.989721959 +0000 UTC m=+2744.024938628" watchObservedRunningTime="2026-09-11 19:01:20.991237389 +0000 UTC m=+2744.026454058"
Sep 11 19:01:22.541042 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:22.540999    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-88vkz_46bfe80b-cd69-4ba9-a7fb-128f20aa12d0/dns/0.log"
Sep 11 19:01:22.566022 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:22.565981    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-88vkz_46bfe80b-cd69-4ba9-a7fb-128f20aa12d0/kube-rbac-proxy/0.log"
Sep 11 19:01:22.738083 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:22.738052    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v9sp7_2a2454c2-8f32-4025-9700-5c3464f05e73/dns-node-resolver/0.log"
Sep 11 19:01:23.312185 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:23.312151    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-68756467d4-gmxt5_5a936883-9636-4760-9260-27a0c9a7731b/registry/0.log"
Sep 11 19:01:23.349445 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:23.349417    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d8828_0d5ec55f-0d86-44fc-adb6-b562c0d6f298/node-ca/0.log"
Sep 11 19:01:24.454936 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:24.454905    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk_88e240e9-074e-483f-a292-0b1dc581efb9/istio-proxy/0.log"
Sep 11 19:01:25.009793 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:25.009753    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-vpmf6_b911bd70-ea94-468f-92b2-4bcf6b156d31/discovery/0.log"
Sep 11 19:01:25.084582 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:25.084555    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dwbw9_e701e4f0-fa92-4cd0-ae56-737249752574/istio-proxy/0.log"
Sep 11 19:01:25.088801 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:25.088776    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dwbw9_e701e4f0-fa92-4cd0-ae56-737249752574/istio-proxy/1.log"
Sep 11 19:01:25.155130 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:25.155098    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lnhlh_7ddd784b-815b-42c0-bf68-667587dda11b/payload-processing/0.log"
Sep 11 19:01:25.178374 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:25.178344    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bf46bd65d-jtnbl_f1d12737-e1eb-4321-9473-e06391ac2642/router/0.log"
Sep 11 19:01:26.127830 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:26.127783    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5dh98_2695bbaf-87ba-4022-8025-295c29f09e55/serve-healthcheck-canary/0.log"
Sep 11 19:01:26.623353 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:26.623300    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-dbst7_566a7849-1b81-4d2d-b3ad-80cc5d8031dc/insights-operator/0.log"
Sep 11 19:01:26.623847 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:26.623815    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-dbst7_566a7849-1b81-4d2d-b3ad-80cc5d8031dc/insights-operator/1.log"
Sep 11 19:01:26.711047 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:26.711020    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hmtmq_9bbc029c-d8f7-46bc-bfe9-a55922054b8b/kube-rbac-proxy/0.log"
Sep 11 19:01:26.731939 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:26.731914    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hmtmq_9bbc029c-d8f7-46bc-bfe9-a55922054b8b/exporter/0.log"
Sep 11 19:01:26.755441 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:26.755412    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hmtmq_9bbc029c-d8f7-46bc-bfe9-a55922054b8b/extractor/0.log"
Sep 11 19:01:27.996086 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:27.996051    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5r55n/perf-node-gather-daemonset-dtcjh"
Sep 11 19:01:29.534617 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:29.534508    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g8gpq_04d7afdf-f4a3-42b8-87e5-0612ac2c48b9/manager/0.log"
Sep 11 19:01:29.650023 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:29.649984    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68b7b6b756-c9p8m_ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b/manager/0.log"
Sep 11 19:01:29.694178 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:29.694150    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68b7b6b756-c9p8m_ba5b00f6-bd4a-405a-9c6f-4e296fa02c2b/copy-manifests/0.log"
Sep 11 19:01:30.943457 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:30.943421    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk_88e240e9-074e-483f-a292-0b1dc581efb9/istio-proxy/0.log"
Sep 11 19:01:31.229999 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:31.229901    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78b8f7c5cb-zcqqs_cb3c6a8c-8ef7-4d06-b178-cdee0a263da1/manager/0.log"
Sep 11 19:01:31.256429 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:31.256394    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-8kcqf_21596dda-2c5c-4160-b93b-1d26af60194c/openshift-lws-operator/0.log"
Sep 11 19:01:31.517850 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:31.517817    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-vpmf6_b911bd70-ea94-468f-92b2-4bcf6b156d31/discovery/0.log"
Sep 11 19:01:31.573726 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:31.573605    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dwbw9_e701e4f0-fa92-4cd0-ae56-737249752574/istio-proxy/0.log"
Sep 11 19:01:31.578389 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:31.578359    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dwbw9_e701e4f0-fa92-4cd0-ae56-737249752574/istio-proxy/1.log"
Sep 11 19:01:31.625369 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:31.625336    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lnhlh_7ddd784b-815b-42c0-bf68-667587dda11b/payload-processing/0.log"
Sep 11 19:01:31.642668 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:31.642638    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bf46bd65d-jtnbl_f1d12737-e1eb-4321-9473-e06391ac2642/router/0.log"
Sep 11 19:01:33.054778 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:33.054698    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rfhrk_88e240e9-074e-483f-a292-0b1dc581efb9/istio-proxy/0.log"
Sep 11 19:01:33.650558 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:33.650504    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-vpmf6_b911bd70-ea94-468f-92b2-4bcf6b156d31/discovery/0.log"
Sep 11 19:01:33.710409 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:33.710367    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dwbw9_e701e4f0-fa92-4cd0-ae56-737249752574/istio-proxy/0.log"
Sep 11 19:01:33.715342 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:33.715317    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dwbw9_e701e4f0-fa92-4cd0-ae56-737249752574/istio-proxy/1.log"
Sep 11 19:01:33.767050 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:33.767021    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lnhlh_7ddd784b-815b-42c0-bf68-667587dda11b/payload-processing/0.log"
Sep 11 19:01:33.783238 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:33.783183    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bf46bd65d-jtnbl_f1d12737-e1eb-4321-9473-e06391ac2642/router/0.log"
Sep 11 19:01:36.514220 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:36.514191    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-psl4z_e6dc24b4-e75b-4a74-a189-7ffb153be3ef/migrator/0.log"
Sep 11 19:01:36.552107 ip-10-0-141-82 kubenswrapper[2578]: I0911 19:01:36.552083    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-psl4z_e6dc24b4-e75b-4a74-a189-7ffb153be3ef/graceful-termination/0.log"