Sep 12 01:44:53.038332 ip-10-0-143-251 systemd[1]: Starting Kubernetes Kubelet...
Sep 12 01:44:53.470563 ip-10-0-143-251 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 01:44:53.470563 ip-10-0-143-251 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 12 01:44:53.470563 ip-10-0-143-251 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 01:44:53.470563 ip-10-0-143-251 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 12 01:44:53.470563 ip-10-0-143-251 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 01:44:53.473202 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.473117    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 12 01:44:53.479086 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479062    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 01:44:53.479086 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479081    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 01:44:53.479086 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479086    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 01:44:53.479086 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479090    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479096    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479103    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479109    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479113    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479117    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479121    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479125    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479130    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479134    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479138    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479142    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479146    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479150    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479153    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479157    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479161    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479165    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479168    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479172    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 01:44:53.479342 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479176    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479180    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479184    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479187    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479191    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479195    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479198    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479203    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479207    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479219    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479224    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479228    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479232    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479237    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479242    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479247    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479252    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479257    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479261    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479265    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 01:44:53.480164 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479270    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479274    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479278    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479282    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479286    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479291    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479295    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479299    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479303    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479308    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479314    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479318    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479322    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479329    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479334    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479338    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479342    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479346    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479349    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 01:44:53.480675 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479353    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479358    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479363    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479367    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479371    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479378    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479385    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479391    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479396    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479401    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479406    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479411    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479415    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479419    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479423    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479429    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479435    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479439    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479444    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 01:44:53.481381 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479448    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479452    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479458    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479463    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479467    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479471    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479476    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479480    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479484    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479488    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479492    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479496    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479500    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479505    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479509    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479513    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479517    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.479521    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481540    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481553    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 01:44:53.482206 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481559    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481564    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481568    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481573    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481577    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481581    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481585    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481590    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481594    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481598    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481602    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481606    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481610    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481614    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481619    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481624    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481629    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481633    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481637    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481642    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 01:44:53.483034 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481646    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481650    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481654    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481658    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481662    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481669    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481676    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481681    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481685    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481690    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481694    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481698    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481702    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481706    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481710    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481715    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481719    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481723    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481727    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 01:44:53.483989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481732    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481736    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481739    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481744    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481749    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481753    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481758    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481762    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481766    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481772    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481776    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481780    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481784    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481789    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481793    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481797    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481803    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481807    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481811    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481818    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 01:44:53.484831 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481822    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481827    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481831    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481835    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481839    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481844    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481848    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481854    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481860    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481864    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481868    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481872    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481877    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481881    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481887    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481891    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481896    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481900    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481904    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 01:44:53.485346 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481908    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481913    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481917    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481922    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481927    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481931    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481935    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481940    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481944    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481948    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481952    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481957    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481963    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481967    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481972    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481976    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481982    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481987    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.481991    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482645    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 01:44:53.485905 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482655    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482660    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482665    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482670    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482677    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482683    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482687    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482691    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482696    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482700    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482704    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482709    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482714    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482718    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482722    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482728    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482732    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482736    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482740    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482744    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 01:44:53.486447 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482749    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482754    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482758    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482763    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482769    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482773    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482777    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482782    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482786    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482790    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482795    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482802    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482808    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482813    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482817    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482822    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482826    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482830    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482837    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482843    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 01:44:53.487060 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482848    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482852    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482857    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482863    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482867    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482872    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482876    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482880    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482885    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482890    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482894    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482898    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482902    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482906    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482910    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482914    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482920    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482924    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482928    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482933    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 01:44:53.487657 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482937    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482941    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482945    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482949    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482952    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482957    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482961    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482965    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482969    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482973    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482977    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482981    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482985    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482989    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482993    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.482999    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483003    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483007    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483011    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483015    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 01:44:53.488213 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483019    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483024    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483029    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483035    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483057    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483062    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483066    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483070    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483076    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483079    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483084    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483088    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483092    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483096    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483100    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483104    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483109    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.483113    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483209    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483263    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 12 01:44:53.488786 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483272    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483279    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483285    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483290    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483297    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483303    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483308    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483315    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483320    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483326    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483330    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483335    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483340    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483345    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483351    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483356    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483364    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483368    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483373    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483378    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483385    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483391    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483396    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 12 01:44:53.489285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483401    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483406    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483410    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483415    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483419    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483424    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483430    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483436    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483441    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483446    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483450    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483455    2580 flags.go:64] FLAG: --enable-server="true"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483460    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483466    2580 flags.go:64] FLAG: --event-burst="100"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483471    2580 flags.go:64] FLAG: --event-qps="50"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483476    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483482    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483487    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483493    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483497    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483502    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483507    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483511    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483516    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483522    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 12 01:44:53.489892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483528    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483533    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483538    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483543    2580 flags.go:64] FLAG: --feature-gates=""
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483552    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483557    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483562    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483567    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483572    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483577    2580 flags.go:64] FLAG: --help="false"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483582    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483587    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483591    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483596    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483602    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483607    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483612    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483617    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483622    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483627    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483632    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483637    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483642    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483648    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 12 01:44:53.490547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483653    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483657    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483662    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483667    2580 flags.go:64] FLAG: --lock-file=""
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483671    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483676    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483681    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483689    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483695    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483700    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483705    2580 flags.go:64] FLAG: --logging-format="text"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483710    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483717    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483721    2580 flags.go:64] FLAG: --manifest-url=""
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483726    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483732    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483737    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483744    2580 flags.go:64] FLAG: --max-pods="110"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483748    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483753    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483758    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483763    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483768    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483773    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483777    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 12 01:44:53.491161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483789    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483794    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483799    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483804    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483809    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483817    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483824    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483829    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483834    2580 flags.go:64] FLAG: --port="10250"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483839    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483843    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0be14958b453ec405"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483849    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483854    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483859    2580 flags.go:64] FLAG: --register-node="true"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483863    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483876    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483882    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483886    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483891    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483897    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483904    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483909    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483914    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483918    2580 flags.go:64] FLAG: --runonce="false"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483923    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 12 01:44:53.491825 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483928    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483933    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483938    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483942    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483947    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483952    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483957    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483961    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483966    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483970    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483975    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483980    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483984    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483991    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.483999    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484003    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484007    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484014    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484019    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484024    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484029    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484034    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484057    2580 flags.go:64] FLAG: --v="2"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484064    2580 flags.go:64] FLAG: --version="false"
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484071    2580 flags.go:64] FLAG: --vmodule=""
Sep 12 01:44:53.492485 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484077    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.484083    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484295    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484303    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484308    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484313    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484317    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484322    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484327    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484331    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484336    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484340    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484344    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484349    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484353    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484357    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484361    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484365    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484370    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484374    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 01:44:53.493180 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484380    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484385    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484389    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484392    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484397    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484401    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484405    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484410    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484414    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484418    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484422    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484426    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484432    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484436    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484441    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484445    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484449    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484455    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484463    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 01:44:53.493705 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484470    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484475    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484480    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484484    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484489    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484494    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484498    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484503    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484507    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484512    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484516    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484520    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484525    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484532    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484537    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484541    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484546    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484550    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484554    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 01:44:53.494178 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484558    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484562    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484567    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484571    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484575    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484579    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484583    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484589    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484593    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484597    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484601    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484605    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484609    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484614    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484618    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484622    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484626    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484631    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484635    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484639    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 01:44:53.494644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484643    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484647    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484650    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484655    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484661    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484667    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484673    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484678    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484682    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484687    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484691    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484695    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484699    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484704    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484708    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484712    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484717    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484722    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484726    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484730    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 01:44:53.495147 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484735    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484739    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.484743    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.485581    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.493250    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.493268    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.494412    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.494421    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494482    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494487    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494491    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494494    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494497    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494500    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494503    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 01:44:53.495623 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494506    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494508    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494511    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494514    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494516    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494519    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494521    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494524    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494526    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494529    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494533    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494537    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494539    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494542    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494545    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494547    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494550    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494553    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494555    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494557    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 01:44:53.495998 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494560    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494563    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494572    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494575    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494578    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494580    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494582    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494585    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494587    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494590    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494592    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494595    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494597    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494600    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494602    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494605    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494607    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494610    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494612    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 01:44:53.496557 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494615    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494617    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494620    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494622    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494625    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494627    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494630    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494632    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494636    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494639    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494642    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494645    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494648    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494651    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494653    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494656    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494665    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494668    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494670    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494673    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 01:44:53.497058 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494676    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494678    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494682    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494684    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494687    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494689    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494691    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494694    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494697    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494699    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494702    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494704    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494707    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494709    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494713    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494716    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494719    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494722    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494725    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494728    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 01:44:53.497537 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494731    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494734    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494736    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494739    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494742    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494744    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494747    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494750    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494752    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494761    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494764    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494766    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494769    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.494774    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494918    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 01:44:53.498013 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494923    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494926    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494929    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494932    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494935    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494938    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494941    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494944    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494947    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494949    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494952    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494955    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494957    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494960    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494962    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494965    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494967    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494970    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494972    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494975    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 01:44:53.498403 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494977    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494980    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494982    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494986    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494990    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494993    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.494996    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495004    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495007    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495009    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495012    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495015    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495017    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495020    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495022    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495025    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495027    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495029    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495032    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495034    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 01:44:53.498891 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495037    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495039    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495057    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495060    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495062    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495065    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495068    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495070    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495072    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495075    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495078    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495080    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495083    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495085    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495088    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495090    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495093    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495096    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495099    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495101    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 01:44:53.499393 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495114    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495117    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495119    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495122    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495124    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495127    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495130    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495132    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495134    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495137    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495139    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495142    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495145    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495147    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495150    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495153    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495156    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495158    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495161    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495164    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 01:44:53.499879 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495166    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495168    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495171    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495173    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495176    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495178    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495182    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495186    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495189    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495193    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495196    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495199    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495202    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495211    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495214    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495216    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495219    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 01:44:53.500387 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:53.495221    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 01:44:53.500789 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.495226    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 01:44:53.500789 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.495351    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 12 01:44:53.500789 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.498403    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 12 01:44:53.500789 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.500118    2580 server.go:1019] "Starting client certificate rotation"
Sep 12 01:44:53.500789 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.500215    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 12 01:44:53.500789 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.500272    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 12 01:44:53.504314 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.504301    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 12 01:44:53.510893 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.510871    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 12 01:44:53.515075 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.515059    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 12 01:44:53.518413 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.518397    2580 log.go:25] "Validated CRI v1 image API"
Sep 12 01:44:53.519219 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.519202    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 12 01:44:53.523435 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.523417    2580 fs.go:135] Filesystem UUIDs: map[790ce223-7450-4f0f-8f0e-c74f913dfa7d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d48a4b32-6aca-4ff2-afef-82c83f6d04df:/dev/nvme0n1p4]
Sep 12 01:44:53.523435 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.523433    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 12 01:44:53.528436 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.528327    2580 manager.go:216] Machine: {Timestamp:2026-09-12 01:44:53.526947628 +0000 UTC m=+0.367015018 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100331 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ae10bf354ebbf1e23168f2cefa947 SystemUUID:ec2ae10b-f354-ebbf-1e23-168f2cefa947 BootID:3a8f83a2-fe6f-46f4-a35f-d4a80d02cb00 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1c:93:e2:51:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1c:93:e2:51:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:ef:cb:75:19:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 12 01:44:53.528436 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.528425    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 12 01:44:53.528545 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.528500    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 12 01:44:53.531735 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.531714    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 12 01:44:53.531895 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.531738    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-251.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 12 01:44:53.532995 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.532982    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 12 01:44:53.533059 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.533001    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 12 01:44:53.533059 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.533001    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 12 01:44:53.533059 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.533010    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 12 01:44:53.533672 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.533661    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 12 01:44:53.534527 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.534515    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 12 01:44:53.538201 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.538188    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 12 01:44:53.538313 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.538304    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 12 01:44:53.538354 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.538331    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 12 01:44:53.538354 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.538348    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 12 01:44:53.538431 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.538382    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 12 01:44:53.538431 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.538392    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 12 01:44:53.538431 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.538402    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 12 01:44:53.538770 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.538757    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 12 01:44:53.538836 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.538776    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 12 01:44:53.540999 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.540973    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 12 01:44:53.544099 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.544071    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 12 01:44:53.544171 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.544123    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 12 01:44:53.546329 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546306    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 12 01:44:53.546541 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546530    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 12 01:44:53.546541 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546541    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 12 01:44:53.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546547    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 12 01:44:53.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546556    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 12 01:44:53.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546562    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 12 01:44:53.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546575    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 12 01:44:53.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546581    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 12 01:44:53.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546587    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 12 01:44:53.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546593    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 12 01:44:53.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546602    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 12 01:44:53.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.546607    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 12 01:44:53.547291 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.547275    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 12 01:44:53.547291 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.547288    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 12 01:44:53.549605 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.549563    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-251.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 12 01:44:53.549722 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.549634    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-251.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 12 01:44:53.549722 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.549629    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 12 01:44:53.551263 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.551127    2580 server.go:1268] "Started kubelet"
Sep 12 01:44:53.551263 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.551213    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 12 01:44:53.551407 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.551215    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 12 01:44:53.551407 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.551364    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 12 01:44:53.552090 ip-10-0-143-251 systemd[1]: Started Kubernetes Kubelet.
Sep 12 01:44:53.552511 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.552496    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 12 01:44:53.553665 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.553640    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 12 01:44:53.556792 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.555894    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-251.ec2.internal.18d46f736f3cb94f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-251.ec2.internal,UID:ip-10-0-143-251.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-251.ec2.internal,},FirstTimestamp:2026-09-12 01:44:53.551102287 +0000 UTC m=+0.391169677,LastTimestamp:2026-09-12 01:44:53.551102287 +0000 UTC m=+0.391169677,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-251.ec2.internal,}"
Sep 12 01:44:53.557136 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.557109    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 12 01:44:53.560094 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.560071    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 12 01:44:53.560679 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.560659    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 12 01:44:53.560867 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.560851    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:53.561309 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561294    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 12 01:44:53.561309 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561309    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 12 01:44:53.561433 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561409    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 12 01:44:53.561433 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561416    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 12 01:44:53.561537 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561507    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 12 01:44:53.561585 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561562    2580 factory.go:55] Registering systemd factory
Sep 12 01:44:53.561585 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561580    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 12 01:44:53.561801 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561787    2580 factory.go:153] Registering CRI-O factory
Sep 12 01:44:53.561858 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561803    2580 factory.go:223] Registration of the crio container factory successfully
Sep 12 01:44:53.561890 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561866    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 12 01:44:53.561890 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561885    2580 factory.go:103] Registering Raw factory
Sep 12 01:44:53.562000 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.561901    2580 manager.go:1196] Started watching for new ooms in manager
Sep 12 01:44:53.562393 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.562380    2580 manager.go:349] Starting recovery of all containers
Sep 12 01:44:53.564315 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.564285    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-251.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 12 01:44:53.564315 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.564306    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 12 01:44:53.574039 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.574023    2580 manager.go:354] Recovery completed
Sep 12 01:44:53.576962 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.576942    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z6d8f"
Sep 12 01:44:53.579674 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.579658    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 01:44:53.581799 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.581783    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 01:44:53.581842 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.581812    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 01:44:53.581842 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.581827    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientPID"
Sep 12 01:44:53.582710 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.582554    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 12 01:44:53.582710 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.582567    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 12 01:44:53.582710 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.582586    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 12 01:44:53.582710 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.582591    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 01:44:53.583396 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.583379    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 01:44:53.583495 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.583407    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 01:44:53.583495 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.583419    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientPID"
Sep 12 01:44:53.583829 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.583742    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-251.ec2.internal.18d46f7371112389  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-251.ec2.internal,UID:ip-10-0-143-251.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-251.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-251.ec2.internal,},FirstTimestamp:2026-09-12 01:44:53.581800329 +0000 UTC m=+0.421867726,LastTimestamp:2026-09-12 01:44:53.581800329 +0000 UTC m=+0.421867726,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-251.ec2.internal,}"
Sep 12 01:44:53.584027 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.583997    2580 policy_none.go:49] "None policy: Start"
Sep 12 01:44:53.584091 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.584060    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 12 01:44:53.584091 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.584076    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 12 01:44:53.584205 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.584149    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z6d8f"
Sep 12 01:44:53.585583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.585568    2580 policy_none.go:47] "Start"
Sep 12 01:44:53.615639 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.615622    2580 manager.go:337] "Starting Device Plugin manager"
Sep 12 01:44:53.626239 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.615663    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 12 01:44:53.626239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.615676    2580 server.go:87] "Starting device plugin registration server"
Sep 12 01:44:53.626239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.615980    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 12 01:44:53.626239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.615991    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 12 01:44:53.626239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.616127    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 12 01:44:53.626239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.616240    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 12 01:44:53.626239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.616250    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 12 01:44:53.626239 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.616702    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 12 01:44:53.626239 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.616749    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:53.657997 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.657972    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 12 01:44:53.659176 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.659159    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 12 01:44:53.659234 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.659217    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 12 01:44:53.659286 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.659247    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 12 01:44:53.659329 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.659294    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 12 01:44:53.661165 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.661146    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 01:44:53.716097 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.716069    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 01:44:53.717111 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.717092    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 01:44:53.717226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.717123    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 01:44:53.717226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.717138    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientPID"
Sep 12 01:44:53.717226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.717161    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.726002 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.725932    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.726002 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.725958    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-251.ec2.internal\": node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:53.747148 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.747129    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:53.760187 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.760163    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal"]
Sep 12 01:44:53.760261 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.760232    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 01:44:53.761711 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.761691    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 01:44:53.761792 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.761724    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 01:44:53.761792 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.761736    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientPID"
Sep 12 01:44:53.761878 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.761860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4357a5787aab4e0adbbb515f5d6ebd04-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal\" (UID: \"4357a5787aab4e0adbbb515f5d6ebd04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.761922 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.761905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4357a5787aab4e0adbbb515f5d6ebd04-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal\" (UID: \"4357a5787aab4e0adbbb515f5d6ebd04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.762955 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.762941    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 01:44:53.763093 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.763080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.763131 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.763111    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 01:44:53.765112 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.765093    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 01:44:53.765194 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.765122    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 01:44:53.765194 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.765132    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientPID"
Sep 12 01:44:53.765194 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.765093    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 01:44:53.765287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.765201    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 01:44:53.765287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.765212    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientPID"
Sep 12 01:44:53.766509 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.766492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.766589 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.766517    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 01:44:53.767198 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.767180    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 01:44:53.767286 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.767207    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 01:44:53.767286 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.767221    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeHasSufficientPID"
Sep 12 01:44:53.799807 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.799783    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-251.ec2.internal\" not found" node="ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.803217 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.803202    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-251.ec2.internal\" not found" node="ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.847210 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.847187    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:53.862393 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.862372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4357a5787aab4e0adbbb515f5d6ebd04-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal\" (UID: \"4357a5787aab4e0adbbb515f5d6ebd04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.862393 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.862383    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4357a5787aab4e0adbbb515f5d6ebd04-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal\" (UID: \"4357a5787aab4e0adbbb515f5d6ebd04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.862515 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.862418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4357a5787aab4e0adbbb515f5d6ebd04-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal\" (UID: \"4357a5787aab4e0adbbb515f5d6ebd04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.862515 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.862458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4357a5787aab4e0adbbb515f5d6ebd04-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal\" (UID: \"4357a5787aab4e0adbbb515f5d6ebd04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:53.947556 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:53.947530    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:53.962809 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:53.962785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8691b2b790f8ef4d7e8578a8215704ee-config\") pod \"kube-apiserver-proxy-ip-10-0-143-251.ec2.internal\" (UID: \"8691b2b790f8ef4d7e8578a8215704ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:54.048294 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:54.048219    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:54.063551 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:54.063527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8691b2b790f8ef4d7e8578a8215704ee-config\") pod \"kube-apiserver-proxy-ip-10-0-143-251.ec2.internal\" (UID: \"8691b2b790f8ef4d7e8578a8215704ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:54.063624 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:54.063562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8691b2b790f8ef4d7e8578a8215704ee-config\") pod \"kube-apiserver-proxy-ip-10-0-143-251.ec2.internal\" (UID: \"8691b2b790f8ef4d7e8578a8215704ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:54.103714 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:54.103672    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:54.106195 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:54.106176    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:54.149191 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:54.149163    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:54.249774 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:54.249724    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:55.663379 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.663350    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:55.763935 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.763905    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-251.ec2.internal\" not found"
Sep 12 01:44:55.812344 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.812315    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 12 01:44:55.812760 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.812445    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 12 01:44:55.850533 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.850511    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 01:44:55.852345 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.852332    2580 apiserver.go:52] "Watching apiserver"
Sep 12 01:44:55.862675 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.862653    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 12 01:44:55.863854 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.863830    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-75ddj","openshift-network-operator/iptables-alerter-jzqgs","kube-system/global-pull-secret-syncer-2cbp5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v","openshift-image-registry/node-ca-zjw29","openshift-multus/multus-5xxn9","openshift-multus/network-metrics-daemon-w2745","openshift-ovn-kubernetes/ovnkube-node-gmpjd","kube-system/konnectivity-agent-vt5cc","openshift-cluster-node-tuning-operator/tuned-ddvxp","openshift-dns/node-resolver-svxhl","openshift-multus/multus-additional-cni-plugins-xl2v2"]
Sep 12 01:44:55.867299 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.867280    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:44:55.867416 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.867344    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:44:55.868371 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.868354    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.869735 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.869717    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:55.869817 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.869774    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:44:55.870549 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.870536    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 12 01:44:55.870669 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.870655    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 12 01:44:55.870715 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.870699    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 12 01:44:55.870813 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.870796    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hsmvm\""
Sep 12 01:44:55.871039 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.871017    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.872359 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.872343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.873277 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.873255    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 12 01:44:55.873426 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.873257    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 12 01:44:55.873426 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.873318    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 12 01:44:55.873426 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.873361    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gc2gn\""
Sep 12 01:44:55.873426 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.873385    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 12 01:44:55.873605 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.873511    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.874237 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.874205    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:55.874330 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.874276    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 12 01:44:55.874385 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.874350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r7n6p\""
Sep 12 01:44:55.874455 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.874442    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 12 01:44:55.874666 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.874653    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 12 01:44:55.874876 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.874861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:55.874944 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.874925    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:44:55.875669 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.875654    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 12 01:44:55.875793 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.875776    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 12 01:44:55.876026 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.876009    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 12 01:44:55.876026 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.876021    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bb27x\""
Sep 12 01:44:55.876167 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.876143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.876647 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.876625    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 12 01:44:55.877409 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.877393    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:44:55.878505 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.878484    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bvbdj\""
Sep 12 01:44:55.878618 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.878510    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 12 01:44:55.878618 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.878493    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 12 01:44:55.878618 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.878560    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 12 01:44:55.878618 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.878611    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 12 01:44:55.878815 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.878564    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 12 01:44:55.878815 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.878636    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 12 01:44:55.878979 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.878967    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.879296 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b590634-a38d-4e33-931d-b7a784f3d07d-dbus\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:55.879364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-cni-dir\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.879364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-socket-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.879364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f58b8bfb-56fb-442f-9400-c8d860f7ffb6-iptables-alerter-script\") pod \"iptables-alerter-jzqgs\" (UID: \"f58b8bfb-56fb-442f-9400-c8d860f7ffb6\") " pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.879458 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlnz8\" (UniqueName: \"kubernetes.io/projected/f58b8bfb-56fb-442f-9400-c8d860f7ffb6-kube-api-access-qlnz8\") pod \"iptables-alerter-jzqgs\" (UID: \"f58b8bfb-56fb-442f-9400-c8d860f7ffb6\") " pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.879458 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879396    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b590634-a38d-4e33-931d-b7a784f3d07d-kubelet-config\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:55.879458 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e050df8-40a2-410e-a291-3177dabbb8f8-host\") pod \"node-ca-zjw29\" (UID: \"0e050df8-40a2-410e-a291-3177dabbb8f8\") " pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.879458 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879423    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-daemon-config\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.879458 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f58b8bfb-56fb-442f-9400-c8d860f7ffb6-host-slash\") pod \"iptables-alerter-jzqgs\" (UID: \"f58b8bfb-56fb-442f-9400-c8d860f7ffb6\") " pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.879597 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0e050df8-40a2-410e-a291-3177dabbb8f8-serviceca\") pod \"node-ca-zjw29\" (UID: \"0e050df8-40a2-410e-a291-3177dabbb8f8\") " pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.879597 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-run-netns\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.879597 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-var-lib-cni-multus\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.879597 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-run-multus-certs\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.879597 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-registration-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.879759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-sys-fs\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.879759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:55.879759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-cnibin\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.879759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e4e4495-a8f4-4a7b-9197-d294e0c25280-cni-binary-copy\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.879759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-hostroot\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.879759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-device-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.879759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.879759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-etc-kubernetes\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq79d\" (UniqueName: \"kubernetes.io/projected/8e4e4495-a8f4-4a7b-9197-d294e0c25280-kube-api-access-fq79d\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879824    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-socket-dir-parent\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-var-lib-kubelet\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfths\" (UniqueName: \"kubernetes.io/projected/9ae26889-09cb-4e31-9af1-4d560513d4af-kube-api-access-nfths\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq2wm\" (UniqueName: \"kubernetes.io/projected/0e050df8-40a2-410e-a291-3177dabbb8f8-kube-api-access-bq2wm\") pod \"node-ca-zjw29\" (UID: \"0e050df8-40a2-410e-a291-3177dabbb8f8\") " pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-var-lib-cni-bin\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8mlz\" (UniqueName: \"kubernetes.io/projected/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-kube-api-access-h8mlz\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879963    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cbpvq\""
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-system-cni-dir\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.880001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.879987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-run-k8s-cni-cncf-io\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.880410 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.880011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-conf-dir\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.880410 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.880119    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 12 01:44:55.880467 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.880444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:55.880917 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.880896    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 12 01:44:55.881040 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.881025    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 12 01:44:55.881267 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.881254    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6krwn\""
Sep 12 01:44:55.881326 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.881285    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 12 01:44:55.881927 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.881910    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:55.882569 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.882550    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 12 01:44:55.882752 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.882738    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wgttv\""
Sep 12 01:44:55.882822 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.882741    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 12 01:44:55.884041 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.884021    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 12 01:44:55.884158 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.884080    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 12 01:44:55.884158 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.884138    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nr4ld\""
Sep 12 01:44:55.885475 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.885459    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 12 01:44:55.890610 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.890087    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 12 01:44:55.890610 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.890183    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal"
Sep 12 01:44:55.890610 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.890238    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal"]
Sep 12 01:44:55.897223 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.897197    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal"]
Sep 12 01:44:55.897333 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.897262    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 12 01:44:55.901366 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.901315    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-11 01:39:54 +0000 UTC" deadline="2028-04-15 17:53:47.407843773 +0000 UTC"
Sep 12 01:44:55.901465 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.901366    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13960h8m51.506481323s"
Sep 12 01:44:55.906764 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.906744    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bvkx8"
Sep 12 01:44:55.914497 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.914481    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bvkx8"
Sep 12 01:44:55.975976 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.975957    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 12 01:44:55.981094 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-run-k8s-cni-cncf-io\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.981192 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.981192 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b590634-a38d-4e33-931d-b7a784f3d07d-dbus\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:55.981192 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-cni-netd\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.981192 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8lrp\" (UniqueName: \"kubernetes.io/projected/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-kube-api-access-t8lrp\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.981192 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-kubernetes\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b590634-a38d-4e33-931d-b7a784f3d07d-dbus\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlnz8\" (UniqueName: \"kubernetes.io/projected/f58b8bfb-56fb-442f-9400-c8d860f7ffb6-kube-api-access-qlnz8\") pod \"iptables-alerter-jzqgs\" (UID: \"f58b8bfb-56fb-442f-9400-c8d860f7ffb6\") " pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-run-k8s-cni-cncf-io\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b590634-a38d-4e33-931d-b7a784f3d07d-kubelet-config\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f58b8bfb-56fb-442f-9400-c8d860f7ffb6-host-slash\") pod \"iptables-alerter-jzqgs\" (UID: \"f58b8bfb-56fb-442f-9400-c8d860f7ffb6\") " pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b590634-a38d-4e33-931d-b7a784f3d07d-kubelet-config\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f58b8bfb-56fb-442f-9400-c8d860f7ffb6-host-slash\") pod \"iptables-alerter-jzqgs\" (UID: \"f58b8bfb-56fb-442f-9400-c8d860f7ffb6\") " pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-run-multus-certs\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/890d8019-49a3-4b97-b6d9-ff60814cb6af-tmp-dir\") pod \"node-resolver-svxhl\" (UID: \"890d8019-49a3-4b97-b6d9-ff60814cb6af\") " pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:55.981459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-run-netns\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-etc-openvswitch\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-registration-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-run-multus-certs\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-sys-fs\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981558    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-sys-fs\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e4e4495-a8f4-4a7b-9197-d294e0c25280-cni-binary-copy\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-registration-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-system-cni-dir\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-slash\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-sysconfig\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-etc-kubernetes\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/890d8019-49a3-4b97-b6d9-ff60814cb6af-hosts-file\") pod \"node-resolver-svxhl\" (UID: \"890d8019-49a3-4b97-b6d9-ff60814cb6af\") " pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:55.981748 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-etc-kubernetes\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-kubelet\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-sysctl-conf\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-var-lib-kubelet\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-socket-dir-parent\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-cni-binary-copy\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-systemd-units\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981920    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-socket-dir-parent\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-run-ovn\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-run-ovn-kubernetes\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-cni-bin\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-modprobe-d\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.981993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:55.982268 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfths\" (UniqueName: \"kubernetes.io/projected/9ae26889-09cb-4e31-9af1-4d560513d4af-kube-api-access-nfths\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tb4q\" (UniqueName: \"kubernetes.io/projected/890d8019-49a3-4b97-b6d9-ff60814cb6af-kube-api-access-8tb4q\") pod \"node-resolver-svxhl\" (UID: \"890d8019-49a3-4b97-b6d9-ff60814cb6af\") " pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.982066    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-cnibin\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-run-systemd\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.982125    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs podName:9ae26889-09cb-4e31-9af1-4d560513d4af nodeName:}" failed. No retries permitted until 2026-09-12 01:44:56.482104764 +0000 UTC m=+2.009005839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs") pod "network-metrics-daemon-w2745" (UID: "9ae26889-09cb-4e31-9af1-4d560513d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-node-log\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-system-cni-dir\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-conf-dir\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e4e4495-a8f4-4a7b-9197-d294e0c25280-cni-binary-copy\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f78vw\" (UniqueName: \"kubernetes.io/projected/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-kube-api-access-f78vw\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-log-socket\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-conf-dir\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-ovnkube-script-lib\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-system-cni-dir\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c-agent-certs\") pod \"konnectivity-agent-vt5cc\" (UID: \"5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c\") " pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:44:55.982934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-sysctl-d\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-cni-dir\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982384    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c-konnectivity-ca\") pod \"konnectivity-agent-vt5cc\" (UID: \"5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c\") " pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-sys\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-host\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-run-openvswitch\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-socket-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f58b8bfb-56fb-442f-9400-c8d860f7ffb6-iptables-alerter-script\") pod \"iptables-alerter-jzqgs\" (UID: \"f58b8bfb-56fb-442f-9400-c8d860f7ffb6\") " pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-cni-dir\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e050df8-40a2-410e-a291-3177dabbb8f8-host\") pod \"node-ca-zjw29\" (UID: \"0e050df8-40a2-410e-a291-3177dabbb8f8\") " pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-daemon-config\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-socket-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982592    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e050df8-40a2-410e-a291-3177dabbb8f8-host\") pod \"node-ca-zjw29\" (UID: \"0e050df8-40a2-410e-a291-3177dabbb8f8\") " pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0e050df8-40a2-410e-a291-3177dabbb8f8-serviceca\") pod \"node-ca-zjw29\" (UID: \"0e050df8-40a2-410e-a291-3177dabbb8f8\") " pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-run-netns\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-run-netns\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.983707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-var-lib-cni-multus\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-lib-modules\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-var-lib-cni-multus\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brqml\" (UniqueName: \"kubernetes.io/projected/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-kube-api-access-brqml\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-var-lib-openvswitch\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-cnibin\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-cnibin\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982946    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0e050df8-40a2-410e-a291-3177dabbb8f8-serviceca\") pod \"node-ca-zjw29\" (UID: \"0e050df8-40a2-410e-a291-3177dabbb8f8\") " pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-hostroot\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8e4e4495-a8f4-4a7b-9197-d294e0c25280-multus-daemon-config\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.982991    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-hostroot\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.982990    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f58b8bfb-56fb-442f-9400-c8d860f7ffb6-iptables-alerter-script\") pod \"iptables-alerter-jzqgs\" (UID: \"f58b8bfb-56fb-442f-9400-c8d860f7ffb6\") " pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-tuned\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-device-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.983064    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret podName:7b590634-a38d-4e33-931d-b7a784f3d07d nodeName:}" failed. No retries permitted until 2026-09-12 01:44:56.483048641 +0000 UTC m=+2.009949723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret") pod "global-pull-secret-syncer-2cbp5" (UID: "7b590634-a38d-4e33-931d-b7a784f3d07d") : object "kube-system"/"original-pull-secret" not registered
Sep 12 01:44:55.984280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-device-dir\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq79d\" (UniqueName: \"kubernetes.io/projected/8e4e4495-a8f4-4a7b-9197-d294e0c25280-kube-api-access-fq79d\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983138    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-ovn-node-metrics-cert\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-tmp\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-var-lib-kubelet\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-ovnkube-config\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-var-lib-kubelet\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-systemd\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq2wm\" (UniqueName: \"kubernetes.io/projected/0e050df8-40a2-410e-a291-3177dabbb8f8-kube-api-access-bq2wm\") pod \"node-ca-zjw29\" (UID: \"0e050df8-40a2-410e-a291-3177dabbb8f8\") " pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-var-lib-cni-bin\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-env-overrides\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e4e4495-a8f4-4a7b-9197-d294e0c25280-host-var-lib-cni-bin\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-run\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:55.985045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.983589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mlz\" (UniqueName: \"kubernetes.io/projected/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-kube-api-access-h8mlz\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.988090 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.988067    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 12 01:44:55.988232 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.988199    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 01:44:55.988311 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.988238    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 01:44:55.988311 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.988252    2580 projected.go:196] Error preparing data for projected volume kube-api-access-4458x for pod openshift-network-diagnostics/network-check-target-75ddj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:44:55.988311 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:55.988308    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x podName:b3a8d7a4-4a7f-474a-8e27-c9a2a4595835 nodeName:}" failed. No retries permitted until 2026-09-12 01:44:56.488293906 +0000 UTC m=+2.015194987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4458x" (UniqueName: "kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x") pod "network-check-target-75ddj" (UID: "b3a8d7a4-4a7f-474a-8e27-c9a2a4595835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:44:55.992509 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.992410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlnz8\" (UniqueName: \"kubernetes.io/projected/f58b8bfb-56fb-442f-9400-c8d860f7ffb6-kube-api-access-qlnz8\") pod \"iptables-alerter-jzqgs\" (UID: \"f58b8bfb-56fb-442f-9400-c8d860f7ffb6\") " pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:55.992611 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.992437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq79d\" (UniqueName: \"kubernetes.io/projected/8e4e4495-a8f4-4a7b-9197-d294e0c25280-kube-api-access-fq79d\") pod \"multus-5xxn9\" (UID: \"8e4e4495-a8f4-4a7b-9197-d294e0c25280\") " pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:55.992611 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.992541    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq2wm\" (UniqueName: \"kubernetes.io/projected/0e050df8-40a2-410e-a291-3177dabbb8f8-kube-api-access-bq2wm\") pod \"node-ca-zjw29\" (UID: \"0e050df8-40a2-410e-a291-3177dabbb8f8\") " pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:55.992611 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.992587    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8mlz\" (UniqueName: \"kubernetes.io/projected/75b124c5-e2b7-4b9d-adfb-022c3659f2e1-kube-api-access-h8mlz\") pod \"aws-ebs-csi-driver-node-mtq5v\" (UID: \"75b124c5-e2b7-4b9d-adfb-022c3659f2e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:55.992611 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:55.992600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfths\" (UniqueName: \"kubernetes.io/projected/9ae26889-09cb-4e31-9af1-4d560513d4af-kube-api-access-nfths\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:56.005959 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.005934    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8691b2b790f8ef4d7e8578a8215704ee.slice/crio-10c6de2f095315606955b581ac2d9672ce3906268b71bd49489447f928452801 WatchSource:0}: Error finding container 10c6de2f095315606955b581ac2d9672ce3906268b71bd49489447f928452801: Status 404 returned error can't find the container with id 10c6de2f095315606955b581ac2d9672ce3906268b71bd49489447f928452801
Sep 12 01:44:56.006233 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.006198    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4357a5787aab4e0adbbb515f5d6ebd04.slice/crio-bc7ab29cb1d148edafb55685a805b5944338b172543a6d034b0cc84e4d361867 WatchSource:0}: Error finding container bc7ab29cb1d148edafb55685a805b5944338b172543a6d034b0cc84e4d361867: Status 404 returned error can't find the container with id bc7ab29cb1d148edafb55685a805b5944338b172543a6d034b0cc84e4d361867
Sep 12 01:44:56.011554 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.011537    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 01:44:56.084847 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.084814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.084847 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.084848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c-konnectivity-ca\") pod \"konnectivity-agent-vt5cc\" (UID: \"5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c\") " pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:44:56.085078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.084868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-sys\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.084909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-host\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.084931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-sys\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.084945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-run-openvswitch\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.084975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-lib-modules\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.084976    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-host\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.084990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brqml\" (UniqueName: \"kubernetes.io/projected/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-kube-api-access-brqml\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-var-lib-openvswitch\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-var-lib-openvswitch\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-tuned\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085094    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-lib-modules\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-run-openvswitch\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-ovn-node-metrics-cert\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-tmp\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-ovnkube-config\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-systemd\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-env-overrides\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-run\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085304    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-cni-netd\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-systemd\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8lrp\" (UniqueName: \"kubernetes.io/projected/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-kube-api-access-t8lrp\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-kubernetes\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/890d8019-49a3-4b97-b6d9-ff60814cb6af-tmp-dir\") pod \"node-resolver-svxhl\" (UID: \"890d8019-49a3-4b97-b6d9-ff60814cb6af\") " pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:56.085513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-run-netns\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-etc-openvswitch\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-run\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c-konnectivity-ca\") pod \"konnectivity-agent-vt5cc\" (UID: \"5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c\") " pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-system-cni-dir\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085533    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-kubernetes\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-slash\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-sysconfig\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-cni-netd\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/890d8019-49a3-4b97-b6d9-ff60814cb6af-hosts-file\") pod \"node-resolver-svxhl\" (UID: \"890d8019-49a3-4b97-b6d9-ff60814cb6af\") " pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-kubelet\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-sysctl-conf\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-var-lib-kubelet\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-cni-binary-copy\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-systemd-units\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.086303 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-run-ovn\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-env-overrides\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-run-ovn-kubernetes\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085823    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/890d8019-49a3-4b97-b6d9-ff60814cb6af-tmp-dir\") pod \"node-resolver-svxhl\" (UID: \"890d8019-49a3-4b97-b6d9-ff60814cb6af\") " pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-cni-bin\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085867    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-system-cni-dir\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-slash\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-modprobe-d\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-run-netns\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085917    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-etc-openvswitch\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-sysconfig\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.085999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/890d8019-49a3-4b97-b6d9-ff60814cb6af-hosts-file\") pod \"node-resolver-svxhl\" (UID: \"890d8019-49a3-4b97-b6d9-ff60814cb6af\") " pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-ovnkube-config\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-sysctl-conf\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-var-lib-kubelet\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-kubelet\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-run-ovn\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-systemd-units\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-run-ovn-kubernetes\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-host-cni-bin\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tb4q\" (UniqueName: \"kubernetes.io/projected/890d8019-49a3-4b97-b6d9-ff60814cb6af-kube-api-access-8tb4q\") pod \"node-resolver-svxhl\" (UID: \"890d8019-49a3-4b97-b6d9-ff60814cb6af\") " pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-cnibin\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-run-systemd\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-node-log\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-cni-binary-copy\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78vw\" (UniqueName: \"kubernetes.io/projected/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-kube-api-access-f78vw\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-log-socket\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-cnibin\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-run-systemd\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-log-socket\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-ovnkube-script-lib\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086534    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-node-log\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.087650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-modprobe-d\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.088158 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c-agent-certs\") pod \"konnectivity-agent-vt5cc\" (UID: \"5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c\") " pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:44:56.088158 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-sysctl-d\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.088158 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.086706    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-sysctl-d\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.088158 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.087040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-ovnkube-script-lib\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.088158 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.087883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-etc-tuned\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.088158 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.087911    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-tmp\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.088490 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.088473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-ovn-node-metrics-cert\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.089179 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.089163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c-agent-certs\") pod \"konnectivity-agent-vt5cc\" (UID: \"5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c\") " pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:44:56.092993 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.092967    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brqml\" (UniqueName: \"kubernetes.io/projected/4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628-kube-api-access-brqml\") pod \"tuned-ddvxp\" (UID: \"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628\") " pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.093240 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.093223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8lrp\" (UniqueName: \"kubernetes.io/projected/44dde392-f421-4b5c-9d4b-e5bc2fb921f8-kube-api-access-t8lrp\") pod \"ovnkube-node-gmpjd\" (UID: \"44dde392-f421-4b5c-9d4b-e5bc2fb921f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.093595 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.093576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tb4q\" (UniqueName: \"kubernetes.io/projected/890d8019-49a3-4b97-b6d9-ff60814cb6af-kube-api-access-8tb4q\") pod \"node-resolver-svxhl\" (UID: \"890d8019-49a3-4b97-b6d9-ff60814cb6af\") " pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:56.096491 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.096475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f78vw\" (UniqueName: \"kubernetes.io/projected/c50a1b1c-0c8c-48b5-bed5-96f6b3591444-kube-api-access-f78vw\") pod \"multus-additional-cni-plugins-xl2v2\" (UID: \"c50a1b1c-0c8c-48b5-bed5-96f6b3591444\") " pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.105508 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.105494    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 01:44:56.202135 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.202075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jzqgs"
Sep 12 01:44:56.212039 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.212015    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58b8bfb_56fb_442f_9400_c8d860f7ffb6.slice/crio-88b7cd781c8207a0cdc55b10a05bbb7daebe1e65c0e0c90d8ed8fcdb7227c504 WatchSource:0}: Error finding container 88b7cd781c8207a0cdc55b10a05bbb7daebe1e65c0e0c90d8ed8fcdb7227c504: Status 404 returned error can't find the container with id 88b7cd781c8207a0cdc55b10a05bbb7daebe1e65c0e0c90d8ed8fcdb7227c504
Sep 12 01:44:56.235106 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.235085    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v"
Sep 12 01:44:56.242996 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.242975    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b124c5_e2b7_4b9d_adfb_022c3659f2e1.slice/crio-fab7b56799dc4621dcbcf153bfeab744fce974b801fce9819c435a4ca91f225a WatchSource:0}: Error finding container fab7b56799dc4621dcbcf153bfeab744fce974b801fce9819c435a4ca91f225a: Status 404 returned error can't find the container with id fab7b56799dc4621dcbcf153bfeab744fce974b801fce9819c435a4ca91f225a
Sep 12 01:44:56.245269 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.245253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zjw29"
Sep 12 01:44:56.249960 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.249941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5xxn9"
Sep 12 01:44:56.252698 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.252677    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e050df8_40a2_410e_a291_3177dabbb8f8.slice/crio-b05186a923e7a8c8fbe1d80af5e81b2e12657948f22f2b59e2e6fc73b20c7fea WatchSource:0}: Error finding container b05186a923e7a8c8fbe1d80af5e81b2e12657948f22f2b59e2e6fc73b20c7fea: Status 404 returned error can't find the container with id b05186a923e7a8c8fbe1d80af5e81b2e12657948f22f2b59e2e6fc73b20c7fea
Sep 12 01:44:56.259055 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.259034    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e4e4495_a8f4_4a7b_9197_d294e0c25280.slice/crio-eb6d3ba9f23e451536b0b66ca4e154ac8465b813cd50ad111e700325acdba0d5 WatchSource:0}: Error finding container eb6d3ba9f23e451536b0b66ca4e154ac8465b813cd50ad111e700325acdba0d5: Status 404 returned error can't find the container with id eb6d3ba9f23e451536b0b66ca4e154ac8465b813cd50ad111e700325acdba0d5
Sep 12 01:44:56.269320 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.269303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:44:56.278291 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.278271    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44dde392_f421_4b5c_9d4b_e5bc2fb921f8.slice/crio-e9477eddde5429b51a829199dd7dc11a80f6c0e78eba90fb6729b9adf8d0e9a9 WatchSource:0}: Error finding container e9477eddde5429b51a829199dd7dc11a80f6c0e78eba90fb6729b9adf8d0e9a9: Status 404 returned error can't find the container with id e9477eddde5429b51a829199dd7dc11a80f6c0e78eba90fb6729b9adf8d0e9a9
Sep 12 01:44:56.287351 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.287331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:44:56.294819 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.294797    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1c0a90_2ed6_4a9e_8673_d8022fc9db0c.slice/crio-9dcfb1e99c2a82bb59236efc87ac26bbe17a53f21774946b6a31c4b4bff10751 WatchSource:0}: Error finding container 9dcfb1e99c2a82bb59236efc87ac26bbe17a53f21774946b6a31c4b4bff10751: Status 404 returned error can't find the container with id 9dcfb1e99c2a82bb59236efc87ac26bbe17a53f21774946b6a31c4b4bff10751
Sep 12 01:44:56.320305 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.320285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ddvxp"
Sep 12 01:44:56.324927 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.324906    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-svxhl"
Sep 12 01:44:56.327928 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.327906    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7dca9a_e8ef_4740_bbcb_eb6a7e0db628.slice/crio-5096e239ea6b91593a8c6990527dc00a35feac2351301d6a9c85648d52d8427d WatchSource:0}: Error finding container 5096e239ea6b91593a8c6990527dc00a35feac2351301d6a9c85648d52d8427d: Status 404 returned error can't find the container with id 5096e239ea6b91593a8c6990527dc00a35feac2351301d6a9c85648d52d8427d
Sep 12 01:44:56.329542 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.329524    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xl2v2"
Sep 12 01:44:56.332398 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.332380    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890d8019_49a3_4b97_b6d9_ff60814cb6af.slice/crio-f612c5228fa041ea0d500531860cdc2ece2a86cfdfa6998f4091f22f3185d656 WatchSource:0}: Error finding container f612c5228fa041ea0d500531860cdc2ece2a86cfdfa6998f4091f22f3185d656: Status 404 returned error can't find the container with id f612c5228fa041ea0d500531860cdc2ece2a86cfdfa6998f4091f22f3185d656
Sep 12 01:44:56.337982 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:44:56.337964    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc50a1b1c_0c8c_48b5_bed5_96f6b3591444.slice/crio-aee5c8639c9c1a35ec5cd1b86a2fde3a0070490af277d22f2a5c1804fe9c3818 WatchSource:0}: Error finding container aee5c8639c9c1a35ec5cd1b86a2fde3a0070490af277d22f2a5c1804fe9c3818: Status 404 returned error can't find the container with id aee5c8639c9c1a35ec5cd1b86a2fde3a0070490af277d22f2a5c1804fe9c3818
Sep 12 01:44:56.399220 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.399182    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 01:44:56.489305 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.489230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:56.489457 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.489307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:56.489457 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.489337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:44:56.489578 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:56.489452    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 01:44:56.489578 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:56.489472    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 01:44:56.489578 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:56.489493    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 01:44:56.489578 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:56.489506    2580 projected.go:196] Error preparing data for projected volume kube-api-access-4458x for pod openshift-network-diagnostics/network-check-target-75ddj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:44:56.489578 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:56.489532    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret podName:7b590634-a38d-4e33-931d-b7a784f3d07d nodeName:}" failed. No retries permitted until 2026-09-12 01:44:57.489514625 +0000 UTC m=+3.016415690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret") pod "global-pull-secret-syncer-2cbp5" (UID: "7b590634-a38d-4e33-931d-b7a784f3d07d") : object "kube-system"/"original-pull-secret" not registered
Sep 12 01:44:56.489578 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:56.489564    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x podName:b3a8d7a4-4a7f-474a-8e27-c9a2a4595835 nodeName:}" failed. No retries permitted until 2026-09-12 01:44:57.489539841 +0000 UTC m=+3.016440911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4458x" (UniqueName: "kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x") pod "network-check-target-75ddj" (UID: "b3a8d7a4-4a7f-474a-8e27-c9a2a4595835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:44:56.489578 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:56.489566    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:44:56.489905 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:56.489623    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs podName:9ae26889-09cb-4e31-9af1-4d560513d4af nodeName:}" failed. No retries permitted until 2026-09-12 01:44:57.489608696 +0000 UTC m=+3.016509759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs") pod "network-metrics-daemon-w2745" (UID: "9ae26889-09cb-4e31-9af1-4d560513d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:44:56.770724 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.770461    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 01:44:56.916758 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.916652    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-11 01:39:55 +0000 UTC" deadline="2028-06-09 11:31:24.685606154 +0000 UTC"
Sep 12 01:44:56.916758 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.916695    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15273h46m27.768916554s"
Sep 12 01:44:56.999000 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:56.998945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal" event={"ID":"8691b2b790f8ef4d7e8578a8215704ee","Type":"ContainerStarted","Data":"10c6de2f095315606955b581ac2d9672ce3906268b71bd49489447f928452801"}
Sep 12 01:44:57.011127 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.011089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-svxhl" event={"ID":"890d8019-49a3-4b97-b6d9-ff60814cb6af","Type":"ContainerStarted","Data":"f612c5228fa041ea0d500531860cdc2ece2a86cfdfa6998f4091f22f3185d656"}
Sep 12 01:44:57.019139 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.019100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ddvxp" event={"ID":"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628","Type":"ContainerStarted","Data":"5096e239ea6b91593a8c6990527dc00a35feac2351301d6a9c85648d52d8427d"}
Sep 12 01:44:57.035903 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.035723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vt5cc" event={"ID":"5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c","Type":"ContainerStarted","Data":"9dcfb1e99c2a82bb59236efc87ac26bbe17a53f21774946b6a31c4b4bff10751"}
Sep 12 01:44:57.060824 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.060708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5xxn9" event={"ID":"8e4e4495-a8f4-4a7b-9197-d294e0c25280","Type":"ContainerStarted","Data":"eb6d3ba9f23e451536b0b66ca4e154ac8465b813cd50ad111e700325acdba0d5"}
Sep 12 01:44:57.072906 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.072840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v" event={"ID":"75b124c5-e2b7-4b9d-adfb-022c3659f2e1","Type":"ContainerStarted","Data":"fab7b56799dc4621dcbcf153bfeab744fce974b801fce9819c435a4ca91f225a"}
Sep 12 01:44:57.079694 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.079666    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jzqgs" event={"ID":"f58b8bfb-56fb-442f-9400-c8d860f7ffb6","Type":"ContainerStarted","Data":"88b7cd781c8207a0cdc55b10a05bbb7daebe1e65c0e0c90d8ed8fcdb7227c504"}
Sep 12 01:44:57.085797 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.085353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl2v2" event={"ID":"c50a1b1c-0c8c-48b5-bed5-96f6b3591444","Type":"ContainerStarted","Data":"aee5c8639c9c1a35ec5cd1b86a2fde3a0070490af277d22f2a5c1804fe9c3818"}
Sep 12 01:44:57.096597 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.096565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" event={"ID":"44dde392-f421-4b5c-9d4b-e5bc2fb921f8","Type":"ContainerStarted","Data":"e9477eddde5429b51a829199dd7dc11a80f6c0e78eba90fb6729b9adf8d0e9a9"}
Sep 12 01:44:57.111864 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.111829    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zjw29" event={"ID":"0e050df8-40a2-410e-a291-3177dabbb8f8","Type":"ContainerStarted","Data":"b05186a923e7a8c8fbe1d80af5e81b2e12657948f22f2b59e2e6fc73b20c7fea"}
Sep 12 01:44:57.123703 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.123675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal" event={"ID":"4357a5787aab4e0adbbb515f5d6ebd04","Type":"ContainerStarted","Data":"bc7ab29cb1d148edafb55685a805b5944338b172543a6d034b0cc84e4d361867"}
Sep 12 01:44:57.497483 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.497367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:57.497483 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.497424    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:44:57.497483 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.497474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:57.497844 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.497610    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 01:44:57.497844 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.497675    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret podName:7b590634-a38d-4e33-931d-b7a784f3d07d nodeName:}" failed. No retries permitted until 2026-09-12 01:44:59.497657087 +0000 UTC m=+5.024558161 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret") pod "global-pull-secret-syncer-2cbp5" (UID: "7b590634-a38d-4e33-931d-b7a784f3d07d") : object "kube-system"/"original-pull-secret" not registered
Sep 12 01:44:57.498125 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.498103    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:44:57.498199 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.498164    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs podName:9ae26889-09cb-4e31-9af1-4d560513d4af nodeName:}" failed. No retries permitted until 2026-09-12 01:44:59.498147669 +0000 UTC m=+5.025048742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs") pod "network-metrics-daemon-w2745" (UID: "9ae26889-09cb-4e31-9af1-4d560513d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:44:57.498298 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.498284    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 01:44:57.498351 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.498308    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 01:44:57.498351 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.498320    2580 projected.go:196] Error preparing data for projected volume kube-api-access-4458x for pod openshift-network-diagnostics/network-check-target-75ddj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:44:57.498464 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.498360    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x podName:b3a8d7a4-4a7f-474a-8e27-c9a2a4595835 nodeName:}" failed. No retries permitted until 2026-09-12 01:44:59.498347602 +0000 UTC m=+5.025248667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4458x" (UniqueName: "kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x") pod "network-check-target-75ddj" (UID: "b3a8d7a4-4a7f-474a-8e27-c9a2a4595835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:44:57.917651 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.917535    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-11 01:39:55 +0000 UTC" deadline="2028-05-26 13:21:36.9302226 +0000 UTC"
Sep 12 01:44:57.917651 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.917600    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14939h36m39.0126265s"
Sep 12 01:44:57.972800 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.972771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:44:57.972976 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.972896    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:44:57.972976 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.972949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:57.973078 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:57.973011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:57.973153 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.973125    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:44:57.973317 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:57.973279    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:44:59.518672 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:59.518633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:59.519148 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:59.518686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:44:59.519148 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:59.518734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:59.519148 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.518876    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 01:44:59.519148 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.518938    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret podName:7b590634-a38d-4e33-931d-b7a784f3d07d nodeName:}" failed. No retries permitted until 2026-09-12 01:45:03.518920922 +0000 UTC m=+9.045821994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret") pod "global-pull-secret-syncer-2cbp5" (UID: "7b590634-a38d-4e33-931d-b7a784f3d07d") : object "kube-system"/"original-pull-secret" not registered
Sep 12 01:44:59.519414 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.519343    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:44:59.519414 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.519396    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs podName:9ae26889-09cb-4e31-9af1-4d560513d4af nodeName:}" failed. No retries permitted until 2026-09-12 01:45:03.519379473 +0000 UTC m=+9.046280535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs") pod "network-metrics-daemon-w2745" (UID: "9ae26889-09cb-4e31-9af1-4d560513d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:44:59.519534 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.519475    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 01:44:59.519534 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.519488    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 01:44:59.519534 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.519502    2580 projected.go:196] Error preparing data for projected volume kube-api-access-4458x for pod openshift-network-diagnostics/network-check-target-75ddj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:44:59.519679 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.519539    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x podName:b3a8d7a4-4a7f-474a-8e27-c9a2a4595835 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:03.519528122 +0000 UTC m=+9.046429183 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4458x" (UniqueName: "kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x") pod "network-check-target-75ddj" (UID: "b3a8d7a4-4a7f-474a-8e27-c9a2a4595835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:44:59.973043 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:59.972993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:44:59.973043 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:59.973008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:44:59.973329 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:44:59.972993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:44:59.973329 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.973119    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:44:59.973329 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.973230    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:44:59.973562 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:44:59.973351    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:01.973491 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:01.973456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:01.974051 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:01.973587    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:01.974051 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:01.973765    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:01.974051 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:01.973864    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:01.974051 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:01.973893    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:01.974051 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:01.973982    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:03.553505 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:03.553468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:03.553535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:03.553595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.553640    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.553655    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.553668    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.553681    2580 projected.go:196] Error preparing data for projected volume kube-api-access-4458x for pod openshift-network-diagnostics/network-check-target-75ddj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.553695    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.553718    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret podName:7b590634-a38d-4e33-931d-b7a784f3d07d nodeName:}" failed. No retries permitted until 2026-09-12 01:45:11.553700492 +0000 UTC m=+17.080601566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret") pod "global-pull-secret-syncer-2cbp5" (UID: "7b590634-a38d-4e33-931d-b7a784f3d07d") : object "kube-system"/"original-pull-secret" not registered
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.553737    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x podName:b3a8d7a4-4a7f-474a-8e27-c9a2a4595835 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:11.553727207 +0000 UTC m=+17.080628269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4458x" (UniqueName: "kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x") pod "network-check-target-75ddj" (UID: "b3a8d7a4-4a7f-474a-8e27-c9a2a4595835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:45:03.553897 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.553752    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs podName:9ae26889-09cb-4e31-9af1-4d560513d4af nodeName:}" failed. No retries permitted until 2026-09-12 01:45:11.553745654 +0000 UTC m=+17.080646712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs") pod "network-metrics-daemon-w2745" (UID: "9ae26889-09cb-4e31-9af1-4d560513d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:45:03.973353 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:03.973271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:03.973514 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.973403    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:03.973514 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:03.973439    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:03.973514 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:03.973456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:03.973675 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.973526    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:03.973675 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:03.973607    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:05.973543 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:05.973503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:05.973543 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:05.973542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:05.974175 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:05.973548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:05.974175 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:05.973646    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:05.974175 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:05.973715    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:05.974175 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:05.973784    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:07.973314 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:07.973279    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:07.973746 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:07.973284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:07.973746 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:07.973411    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:07.973746 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:07.973286    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:07.973746 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:07.973479    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:07.973746 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:07.973564    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:09.973086 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:09.973047    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:09.973086 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:09.973079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:09.973629 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:09.973195    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:09.973629 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:09.973228    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:09.973629 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:09.973314    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:09.973629 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:09.973400    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:11.612497 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:11.612459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:11.612534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:11.612555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.612629    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.612683    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.612700    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret podName:7b590634-a38d-4e33-931d-b7a784f3d07d nodeName:}" failed. No retries permitted until 2026-09-12 01:45:27.612681069 +0000 UTC m=+33.139582132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret") pod "global-pull-secret-syncer-2cbp5" (UID: "7b590634-a38d-4e33-931d-b7a784f3d07d") : object "kube-system"/"original-pull-secret" not registered
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.612703    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.612718    2580 projected.go:196] Error preparing data for projected volume kube-api-access-4458x for pod openshift-network-diagnostics/network-check-target-75ddj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.612629    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.612771    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs podName:9ae26889-09cb-4e31-9af1-4d560513d4af nodeName:}" failed. No retries permitted until 2026-09-12 01:45:27.612752 +0000 UTC m=+33.139653075 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs") pod "network-metrics-daemon-w2745" (UID: "9ae26889-09cb-4e31-9af1-4d560513d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:45:11.612953 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.612789    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x podName:b3a8d7a4-4a7f-474a-8e27-c9a2a4595835 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:27.612778271 +0000 UTC m=+33.139679329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4458x" (UniqueName: "kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x") pod "network-check-target-75ddj" (UID: "b3a8d7a4-4a7f-474a-8e27-c9a2a4595835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:45:11.973507 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:11.973425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:11.973705 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:11.973437    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:11.973705 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:11.973576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:11.973705 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.973550    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:11.973705 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.973682    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:11.973879 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:11.973768    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:13.973064 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:13.973025    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:13.973064 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:13.973042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:13.973718 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:13.973025    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:13.973718 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:13.973143    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:13.973718 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:13.973230    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:13.973718 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:13.973312    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:15.186968 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.185055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ddvxp" event={"ID":"4a7dca9a-e8ef-4740-bbcb-eb6a7e0db628","Type":"ContainerStarted","Data":"91135a2ce4fa2a52b0ecb3641e7355c5640b461c7444e3b561362648fcd826ed"}
Sep 12 01:45:15.189472 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.189447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5xxn9" event={"ID":"8e4e4495-a8f4-4a7b-9197-d294e0c25280","Type":"ContainerStarted","Data":"128a89039fc3664188e628a69c73c628f71d1923f7a350626070f00ee97d8757"}
Sep 12 01:45:15.194651 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.194628    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" event={"ID":"44dde392-f421-4b5c-9d4b-e5bc2fb921f8","Type":"ContainerStarted","Data":"a97f14949d42408ac5fb0521087dbfb4629f683bebba4f2ce94ac2882292585f"}
Sep 12 01:45:15.194750 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.194701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" event={"ID":"44dde392-f421-4b5c-9d4b-e5bc2fb921f8","Type":"ContainerStarted","Data":"8d6565bd7ff745b893c346e9045350fb78938fb652b6551c2ec4f14dc56c16fa"}
Sep 12 01:45:15.194807 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.194752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" event={"ID":"44dde392-f421-4b5c-9d4b-e5bc2fb921f8","Type":"ContainerStarted","Data":"b71fae2b809815ed9b3edc7e9b8fc7e76ceef54f59764d0ba09ac8a7380c35be"}
Sep 12 01:45:15.197337 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.197303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal" event={"ID":"8691b2b790f8ef4d7e8578a8215704ee","Type":"ContainerStarted","Data":"ee5e429feca3be83af209fc3f243df0603c3309ffd6fc63e842077dadebae50e"}
Sep 12 01:45:15.205648 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.205563    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ddvxp" podStartSLOduration=1.874324353 podStartE2EDuration="20.205542647s" totalImagesPullingTime="18.331218294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:15.204475307 +0000 UTC m=+20.731376388" watchObservedRunningTime="2026-09-12 01:45:15.205542647 +0000 UTC m=+20.732443731"
Sep 12 01:45:15.230262 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.230202    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-251.ec2.internal" podStartSLOduration=20.230188609 podStartE2EDuration="20.230188609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:15.229873559 +0000 UTC m=+20.756774658" watchObservedRunningTime="2026-09-12 01:45:15.230188609 +0000 UTC m=+20.757089689"
Sep 12 01:45:15.250331 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.250281    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5xxn9" podStartSLOduration=1.6385411570000001 podStartE2EDuration="20.250260952s" totalImagesPullingTime="18.611719795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:15.250059585 +0000 UTC m=+20.776960666" watchObservedRunningTime="2026-09-12 01:45:15.250260952 +0000 UTC m=+20.777162033"
Sep 12 01:45:15.973098 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.973014    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:15.973253 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.973030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:15.973253 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:15.973134    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:15.973253 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:15.973146    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:15.973413 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:15.973274    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:15.973413 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:15.973390    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:16.201593 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.201545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v" event={"ID":"75b124c5-e2b7-4b9d-adfb-022c3659f2e1","Type":"ContainerStarted","Data":"4c24eed02797bd208b752c8e0939f1ff3c7fa3660d695e82210025dc0d0c99de"}
Sep 12 01:45:16.203464 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.203403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jzqgs" event={"ID":"f58b8bfb-56fb-442f-9400-c8d860f7ffb6","Type":"ContainerStarted","Data":"c751724e654ae1558abe901e3339bf4960316be7f06bfe21082f0e7585a45a47"}
Sep 12 01:45:16.205109 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.205072    2580 generic.go:356] "Generic (PLEG): container finished" podID="c50a1b1c-0c8c-48b5-bed5-96f6b3591444" containerID="f662a258529dbd9f1890d467078442c9c123ff6adb296e71c525f94aa0d985a7" exitCode=0
Sep 12 01:45:16.205234 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.205157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl2v2" event={"ID":"c50a1b1c-0c8c-48b5-bed5-96f6b3591444","Type":"ContainerDied","Data":"f662a258529dbd9f1890d467078442c9c123ff6adb296e71c525f94aa0d985a7"}
Sep 12 01:45:16.208932 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.208912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" event={"ID":"44dde392-f421-4b5c-9d4b-e5bc2fb921f8","Type":"ContainerStarted","Data":"cbbf3307a446c400ecf2f3f1eaefea6ece50f3dba062feb1abdc5a371a1c5439"}
Sep 12 01:45:16.209000 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.208986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" event={"ID":"44dde392-f421-4b5c-9d4b-e5bc2fb921f8","Type":"ContainerStarted","Data":"fa827f5ae66d132a84397ddf6e558037f11186e910867711488af35f214c6c89"}
Sep 12 01:45:16.209048 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.209036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" event={"ID":"44dde392-f421-4b5c-9d4b-e5bc2fb921f8","Type":"ContainerStarted","Data":"381ba49c24d60b34e186226acc25f52bd61c34202afebcb2f31f9ebb8ec99b24"}
Sep 12 01:45:16.210878 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.210856    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zjw29" event={"ID":"0e050df8-40a2-410e-a291-3177dabbb8f8","Type":"ContainerStarted","Data":"37bda734ac7b32cc269d08b05420ffe07deff82f5cc282b298e75d9f99962cd9"}
Sep 12 01:45:16.212717 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.212690    2580 generic.go:356] "Generic (PLEG): container finished" podID="4357a5787aab4e0adbbb515f5d6ebd04" containerID="798ed92fb38cb0223d52855df540a6a2bfdd7ca37d5f2c90f76c845234adc273" exitCode=0
Sep 12 01:45:16.212804 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.212724    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal" event={"ID":"4357a5787aab4e0adbbb515f5d6ebd04","Type":"ContainerDied","Data":"798ed92fb38cb0223d52855df540a6a2bfdd7ca37d5f2c90f76c845234adc273"}
Sep 12 01:45:16.214617 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.214593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-svxhl" event={"ID":"890d8019-49a3-4b97-b6d9-ff60814cb6af","Type":"ContainerStarted","Data":"bfccd0057a2f0aff7a546eee737f7b91e98a912b386099d175ee18621efe3e50"}
Sep 12 01:45:16.216403 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.216341    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vt5cc" event={"ID":"5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c","Type":"ContainerStarted","Data":"62359f6487c199f72c97d7e7016d43007eb302c6cfb1ef112c94e4940af9ceab"}
Sep 12 01:45:16.219203 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.219157    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jzqgs" podStartSLOduration=2.774962522 podStartE2EDuration="21.219144658s" totalImagesPullingTime="18.444182136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:16.218932624 +0000 UTC m=+21.745833706" watchObservedRunningTime="2026-09-12 01:45:16.219144658 +0000 UTC m=+21.746045738"
Sep 12 01:45:16.232724 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.232662    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-svxhl" podStartSLOduration=2.932737802 podStartE2EDuration="21.232650272s" totalImagesPullingTime="18.29991247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:16.232249474 +0000 UTC m=+21.759150554" watchObservedRunningTime="2026-09-12 01:45:16.232650272 +0000 UTC m=+21.759551352"
Sep 12 01:45:16.244628 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.244595    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zjw29" podStartSLOduration=2.842541184 podStartE2EDuration="21.244584799s" totalImagesPullingTime="18.402043615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:16.244282153 +0000 UTC m=+21.771183233" watchObservedRunningTime="2026-09-12 01:45:16.244584799 +0000 UTC m=+21.771485879"
Sep 12 01:45:16.292892 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.292831    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vt5cc" podStartSLOduration=2.931000799 podStartE2EDuration="21.292818274s" totalImagesPullingTime="18.361817475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:16.2927838 +0000 UTC m=+21.819684879" watchObservedRunningTime="2026-09-12 01:45:16.292818274 +0000 UTC m=+21.819719354"
Sep 12 01:45:16.553532 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.553360    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 12 01:45:16.941953 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.941800    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-12T01:45:16.553529609Z","UUID":"4af246ab-43a8-4826-bf7b-36e6f1940953","Handler":null,"Name":"","Endpoint":""}
Sep 12 01:45:16.943338 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.943317    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 12 01:45:16.943460 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:16.943352    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 12 01:45:17.220618 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:17.220538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal" event={"ID":"4357a5787aab4e0adbbb515f5d6ebd04","Type":"ContainerStarted","Data":"05978e8a35ab77afed1c75398982741b5ef1d1299ce5e31c8642be5feea3a88a"}
Sep 12 01:45:17.222787 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:17.222754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v" event={"ID":"75b124c5-e2b7-4b9d-adfb-022c3659f2e1","Type":"ContainerStarted","Data":"ae3bb8fd9fa5976c0370902614b20089922738b96920ddccd7f88109e7f7ecbc"}
Sep 12 01:45:17.235160 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:17.235112    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-251.ec2.internal" podStartSLOduration=22.23509411 podStartE2EDuration="22.23509411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:17.23467452 +0000 UTC m=+22.761575600" watchObservedRunningTime="2026-09-12 01:45:17.23509411 +0000 UTC m=+22.761995192"
Sep 12 01:45:17.973404 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:17.973373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:17.973544 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:17.973373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:17.973544 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:17.973501    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:17.973544 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:17.973373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:17.973653 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:17.973543    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:17.973653 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:17.973613    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:18.227880 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:18.227794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v" event={"ID":"75b124c5-e2b7-4b9d-adfb-022c3659f2e1","Type":"ContainerStarted","Data":"e24c0896399fdcb5058a4ec66b1ed8bf18b3cb941b6b8b2e6614818f230c7391"}
Sep 12 01:45:18.232036 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:18.232008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" event={"ID":"44dde392-f421-4b5c-9d4b-e5bc2fb921f8","Type":"ContainerStarted","Data":"322c19a56cf76e79f960e7cf59732e9af24f2df168fa5a3906604be9ac834338"}
Sep 12 01:45:18.244828 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:18.244780    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mtq5v" podStartSLOduration=2.524383189 podStartE2EDuration="23.244765552s" totalImagesPullingTime="20.720382363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:18.243546442 +0000 UTC m=+23.770447522" watchObservedRunningTime="2026-09-12 01:45:18.244765552 +0000 UTC m=+23.771666632"
Sep 12 01:45:19.973588 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:19.973332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:19.973957 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:19.973371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:19.973957 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:19.973649    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:19.973957 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:19.973373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:19.973957 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:19.973730    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:19.973957 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:19.973822    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:20.240342 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:20.240262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" event={"ID":"44dde392-f421-4b5c-9d4b-e5bc2fb921f8","Type":"ContainerStarted","Data":"2188baeba5ba3f6debebe7db270fd53ead00a1775d3d99d43b8ce657c9b7c009"}
Sep 12 01:45:20.240695 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:20.240591    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:45:20.240695 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:20.240666    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:45:20.256812 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:20.256783    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:45:20.257372 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:20.257351    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:45:20.268679 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:20.268640    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd" podStartSLOduration=6.853174179 podStartE2EDuration="25.268623431s" totalImagesPullingTime="18.415449252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:20.268291981 +0000 UTC m=+25.795193061" watchObservedRunningTime="2026-09-12 01:45:20.268623431 +0000 UTC m=+25.795524526"
Sep 12 01:45:20.583286 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:20.583250    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:45:20.583849 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:20.583820    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:45:21.243893 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:21.243584    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:45:21.243893 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:21.243630    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:45:21.243893 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:21.243701    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vt5cc"
Sep 12 01:45:21.574451 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:21.574363    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w2745"]
Sep 12 01:45:21.574590 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:21.574541    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:21.574700 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:21.574675    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:21.574982 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:21.574960    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2cbp5"]
Sep 12 01:45:21.575070 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:21.575058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:21.575163 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:21.575142    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:21.576690 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:21.576655    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-75ddj"]
Sep 12 01:45:21.576802 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:21.576757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:21.576865 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:21.576850    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:22.246425 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:22.246392    2580 generic.go:356] "Generic (PLEG): container finished" podID="c50a1b1c-0c8c-48b5-bed5-96f6b3591444" containerID="5811a80b6f5ed2ce034c996bc79ac7729b9308a501a0e0ee85a816c03715f45c" exitCode=0
Sep 12 01:45:22.246913 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:22.246473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl2v2" event={"ID":"c50a1b1c-0c8c-48b5-bed5-96f6b3591444","Type":"ContainerDied","Data":"5811a80b6f5ed2ce034c996bc79ac7729b9308a501a0e0ee85a816c03715f45c"}
Sep 12 01:45:22.972726 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:22.972687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:22.972891 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:22.972814    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:22.972891 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:22.972867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:22.973002 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:22.972954    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:23.973143 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:23.973113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:23.973615 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:23.973228    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:24.252877 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:24.252848    2580 generic.go:356] "Generic (PLEG): container finished" podID="c50a1b1c-0c8c-48b5-bed5-96f6b3591444" containerID="8d88c66c5da98761700fa9ecd93c3fb887bec2b517871559644a69d3336867ca" exitCode=0
Sep 12 01:45:24.253023 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:24.252897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl2v2" event={"ID":"c50a1b1c-0c8c-48b5-bed5-96f6b3591444","Type":"ContainerDied","Data":"8d88c66c5da98761700fa9ecd93c3fb887bec2b517871559644a69d3336867ca"}
Sep 12 01:45:24.973646 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:24.973463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:24.974045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:24.973539    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:24.974045 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:24.973729    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:24.974045 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:24.973817    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:25.973041 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:25.973012    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:25.973206 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:25.973119    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2cbp5" podUID="7b590634-a38d-4e33-931d-b7a784f3d07d"
Sep 12 01:45:26.260299 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:26.260264    2580 generic.go:356] "Generic (PLEG): container finished" podID="c50a1b1c-0c8c-48b5-bed5-96f6b3591444" containerID="db3426cea888cb5c5a1e63aeb04bf1b21dd50b7dbc27bcb37f93590846a847fe" exitCode=0
Sep 12 01:45:26.260742 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:26.260321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl2v2" event={"ID":"c50a1b1c-0c8c-48b5-bed5-96f6b3591444","Type":"ContainerDied","Data":"db3426cea888cb5c5a1e63aeb04bf1b21dd50b7dbc27bcb37f93590846a847fe"}
Sep 12 01:45:26.973329 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:26.973290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:26.973329 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:26.973326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:26.973592 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:26.973416    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2745" podUID="9ae26889-09cb-4e31-9af1-4d560513d4af"
Sep 12 01:45:26.973592 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:26.973520    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-75ddj" podUID="b3a8d7a4-4a7f-474a-8e27-c9a2a4595835"
Sep 12 01:45:27.633580 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.633494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:27.633580 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.633532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:27.633580 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.633567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:27.634080 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:27.633654    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 01:45:27.634080 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:27.633708    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret podName:7b590634-a38d-4e33-931d-b7a784f3d07d nodeName:}" failed. No retries permitted until 2026-09-12 01:45:59.633691027 +0000 UTC m=+65.160592101 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret") pod "global-pull-secret-syncer-2cbp5" (UID: "7b590634-a38d-4e33-931d-b7a784f3d07d") : object "kube-system"/"original-pull-secret" not registered
Sep 12 01:45:27.634080 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:27.633653    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:45:27.634080 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:27.633720    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 01:45:27.634080 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:27.633737    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 01:45:27.634080 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:27.633750    2580 projected.go:196] Error preparing data for projected volume kube-api-access-4458x for pod openshift-network-diagnostics/network-check-target-75ddj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:45:27.634080 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:27.633739    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs podName:9ae26889-09cb-4e31-9af1-4d560513d4af nodeName:}" failed. No retries permitted until 2026-09-12 01:45:59.633733023 +0000 UTC m=+65.160634081 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs") pod "network-metrics-daemon-w2745" (UID: "9ae26889-09cb-4e31-9af1-4d560513d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 01:45:27.634080 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:27.633799    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x podName:b3a8d7a4-4a7f-474a-8e27-c9a2a4595835 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:59.633787265 +0000 UTC m=+65.160688328 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4458x" (UniqueName: "kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x") pod "network-check-target-75ddj" (UID: "b3a8d7a4-4a7f-474a-8e27-c9a2a4595835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 01:45:27.837015 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.836984    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-251.ec2.internal" event="NodeReady"
Sep 12 01:45:27.837179 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.837118    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 12 01:45:27.870542 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.870515    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c96f7fd56-v4sbh"]
Sep 12 01:45:27.890056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.889992    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-9fnqn"]
Sep 12 01:45:27.890166 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.890132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:27.896667 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.896621    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 12 01:45:27.896868 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.896712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 12 01:45:27.896868 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.896744    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2s9js\""
Sep 12 01:45:27.896868 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.896621    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 12 01:45:27.902915 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.902880    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 12 01:45:27.909469 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.909445    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"]
Sep 12 01:45:27.909603 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.909571    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:27.912604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.911933    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 12 01:45:27.912604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.912242    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 12 01:45:27.912604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.912275    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 12 01:45:27.912604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.912324    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 12 01:45:27.912604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.912557    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zntwk\""
Sep 12 01:45:27.917980 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.917822    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 12 01:45:27.933904 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.933880    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-76f959c4b4-5lb49"]
Sep 12 01:45:27.933904 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.934042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:27.937386 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.937367    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 12 01:45:27.937601 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.937580    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 12 01:45:27.937929 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.937908    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 12 01:45:27.938020 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.937976    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5jwvz\""
Sep 12 01:45:27.949142 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.949124    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 12 01:45:27.949868 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.949845    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj"]
Sep 12 01:45:27.949970 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.949950    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:27.957239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.957199    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 12 01:45:27.957801 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.957761    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 12 01:45:27.958028 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.958009    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-txvrz\""
Sep 12 01:45:27.959636 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.959616    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 12 01:45:27.959729 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.959635    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 12 01:45:27.963002 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.962986    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 12 01:45:27.963559 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.963541    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 12 01:45:27.968188 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.968171    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"]
Sep 12 01:45:27.968298 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.968289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj"
Sep 12 01:45:27.970924 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.970904    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-g9djf\""
Sep 12 01:45:27.971026 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.970928    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 12 01:45:27.971800 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.971779    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 12 01:45:27.990285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.990264    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-m6nx8"]
Sep 12 01:45:27.990403 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.990386    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:27.993437 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.993415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vwpzs\""
Sep 12 01:45:27.996270 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.996241    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 12 01:45:27.996270 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.996258    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 12 01:45:27.996413 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:27.996347    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 12 01:45:28.012360 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.012341    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c96f7fd56-v4sbh"]
Sep 12 01:45:28.012447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.012392    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"]
Sep 12 01:45:28.012447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.012419    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-9fnqn"]
Sep 12 01:45:28.012447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.012439    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"]
Sep 12 01:45:28.012604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.012452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.012604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.012458    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-m6nx8"]
Sep 12 01:45:28.012604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.012535    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76f959c4b4-5lb49"]
Sep 12 01:45:28.012604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.012563    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"]
Sep 12 01:45:28.012788 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.012614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:28.016473 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.016454    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 12 01:45:28.016780 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.016762    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 12 01:45:28.017063 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.017042    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 12 01:45:28.017445 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.017428    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xrn54\""
Sep 12 01:45:28.017784 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.017765    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 12 01:45:28.018046 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.018029    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 12 01:45:28.022456 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.022437    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 12 01:45:28.032361 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.032341    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"]
Sep 12 01:45:28.032468 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.032453    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.035631 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035611    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-mn5fg\""
Sep 12 01:45:28.035777 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035758    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 12 01:45:28.035865 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035805    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 12 01:45:28.035865 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035829    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 12 01:45:28.035865 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/417f469c-1b81-454a-b5a1-bdd3a402ddbc-tmp\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.036022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/417f469c-1b81-454a-b5a1-bdd3a402ddbc-snapshots\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.036022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035917    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 12 01:45:28.036022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/417f469c-1b81-454a-b5a1-bdd3a402ddbc-serving-cert\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.036022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-bound-sa-token\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.036022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.035986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vzj9\" (UniqueName: \"kubernetes.io/projected/59059329-6a69-4148-8545-54cb110614d6-kube-api-access-9vzj9\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.036022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj47h\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-kube-api-access-cj47h\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/417f469c-1b81-454a-b5a1-bdd3a402ddbc-trusted-ca-bundle\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-installation-pull-secrets\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-registry-certificates\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr92n\" (UniqueName: \"kubernetes.io/projected/417f469c-1b81-454a-b5a1-bdd3a402ddbc-kube-api-access-pr92n\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036187    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-default-certificate\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/417f469c-1b81-454a-b5a1-bdd3a402ddbc-service-ca-bundle\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.036287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-stats-auth\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.036582 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-image-registry-private-configuration\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.036582 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-trusted-ca\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.036582 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:28.036582 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.036582 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865c063b-707c-4239-b057-6194b27e3929-ca-trust-extracted\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.036582 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.036478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9fhk\" (UniqueName: \"kubernetes.io/projected/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-kube-api-access-f9fhk\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:28.058157 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.058140    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-x49c5"]
Sep 12 01:45:28.058696 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.058679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.061191 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.061172    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 12 01:45:28.061312 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.061291    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 12 01:45:28.061473 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.061381    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 12 01:45:28.061473 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.061390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 12 01:45:28.061473 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.061429    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-dctbt\""
Sep 12 01:45:28.084691 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.084668    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj"]
Sep 12 01:45:28.084787 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.084708    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cqnvn"]
Sep 12 01:45:28.084854 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.084789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x49c5"
Sep 12 01:45:28.087298 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.087280    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-n2qgv\""
Sep 12 01:45:28.088112 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.088090    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 12 01:45:28.088487 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.088470    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 12 01:45:28.105110 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.105088    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lxb7h"]
Sep 12 01:45:28.105226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.105188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cqnvn"
Sep 12 01:45:28.107437 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.107418    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 12 01:45:28.107545 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.107484    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 12 01:45:28.107659 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.107486    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 12 01:45:28.107659 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.107627    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5l94s\""
Sep 12 01:45:28.129223 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.129190    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-x49c5"]
Sep 12 01:45:28.129341 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.129243    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"]
Sep 12 01:45:28.129341 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.129272    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cqnvn"]
Sep 12 01:45:28.129341 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.129290    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lxb7h"]
Sep 12 01:45:28.129341 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.129313    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"]
Sep 12 01:45:28.129591 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.129407    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.131908 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.131889    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 12 01:45:28.132049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.131995    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 12 01:45:28.132049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.132005    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gmqgn\""
Sep 12 01:45:28.136737 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.136707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr92n\" (UniqueName: \"kubernetes.io/projected/417f469c-1b81-454a-b5a1-bdd3a402ddbc-kube-api-access-pr92n\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.136850 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.136745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7042ae8-6999-4ac4-beb3-392bb871dc4c-config\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.136850 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.136779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-default-certificate\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.136850 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.136805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/417f469c-1b81-454a-b5a1-bdd3a402ddbc-service-ca-bundle\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.136850 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.136832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-stats-auth\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.137049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.136855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7042ae8-6999-4ac4-beb3-392bb871dc4c-serving-cert\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.137049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.136881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmxlp\" (UniqueName: \"kubernetes.io/projected/a7042ae8-6999-4ac4-beb3-392bb871dc4c-kube-api-access-rmxlp\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.137049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.136931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-image-registry-private-configuration\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.137049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.136970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865c063b-707c-4239-b057-6194b27e3929-ca-trust-extracted\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.137049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vzj9\" (UniqueName: \"kubernetes.io/projected/59059329-6a69-4148-8545-54cb110614d6-kube-api-access-9vzj9\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.137049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-trusted-ca\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.137364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9fhk\" (UniqueName: \"kubernetes.io/projected/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-kube-api-access-f9fhk\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:28.137364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/417f469c-1b81-454a-b5a1-bdd3a402ddbc-serving-cert\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.137364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-bound-sa-token\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.137364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fprdg\" (UniqueName: \"kubernetes.io/projected/bf43f636-07c2-4d7a-99a3-1307fa989e36-kube-api-access-fprdg\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:28.137364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7271188a-6d22-42bb-acd9-f98490da02c8-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-j2m4q\" (UID: \"7271188a-6d22-42bb-acd9-f98490da02c8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.137364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj47h\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-kube-api-access-cj47h\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.137364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7271188a-6d22-42bb-acd9-f98490da02c8-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-j2m4q\" (UID: \"7271188a-6d22-42bb-acd9-f98490da02c8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137403    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/417f469c-1b81-454a-b5a1-bdd3a402ddbc-service-ca-bundle\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137413    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865c063b-707c-4239-b057-6194b27e3929-ca-trust-extracted\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-installation-pull-secrets\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6nr7\" (UniqueName: \"kubernetes.io/projected/96d615cb-9d90-4ed9-b005-93bbc0f60f94-kube-api-access-s6nr7\") pod \"volume-data-source-validator-86695b7cc7-hg7mj\" (UID: \"96d615cb-9d90-4ed9-b005-93bbc0f60f94\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7042ae8-6999-4ac4-beb3-392bb871dc4c-trusted-ca\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ab6ae3e-2127-401a-aef3-e84336a9cefe-serving-cert\") pod \"service-ca-operator-cf7c776b7-qtdkj\" (UID: \"1ab6ae3e-2127-401a-aef3-e84336a9cefe\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.137686 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/417f469c-1b81-454a-b5a1-bdd3a402ddbc-tmp\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/417f469c-1b81-454a-b5a1-bdd3a402ddbc-snapshots\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.137781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.137918    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.137992    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls podName:2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:28.637974276 +0000 UTC m=+34.164875355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhx4j" (UID: "2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96") : secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.138022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/417f469c-1b81-454a-b5a1-bdd3a402ddbc-trusted-ca-bundle\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.138054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5b6k\" (UniqueName: \"kubernetes.io/projected/7271188a-6d22-42bb-acd9-f98490da02c8-kube-api-access-r5b6k\") pod \"kube-storage-version-migrator-operator-6c8f84f897-j2m4q\" (UID: \"7271188a-6d22-42bb-acd9-f98490da02c8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.138083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ab6ae3e-2127-401a-aef3-e84336a9cefe-config\") pod \"service-ca-operator-cf7c776b7-qtdkj\" (UID: \"1ab6ae3e-2127-401a-aef3-e84336a9cefe\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.138114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-registry-certificates\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.138300 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.138167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8drz2\" (UniqueName: \"kubernetes.io/projected/1ab6ae3e-2127-401a-aef3-e84336a9cefe-kube-api-access-8drz2\") pod \"service-ca-operator-cf7c776b7-qtdkj\" (UID: \"1ab6ae3e-2127-401a-aef3-e84336a9cefe\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.138767 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.138318    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:28.638302876 +0000 UTC m=+34.165203943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : configmap references non-existent config key: service-ca.crt
Sep 12 01:45:28.138834 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.138777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-trusted-ca\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.138892 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.138878    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 01:45:28.138892 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.138890    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c96f7fd56-v4sbh: secret "image-registry-tls" not found
Sep 12 01:45:28.138997 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.138943    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls podName:865c063b-707c-4239-b057-6194b27e3929 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:28.638924766 +0000 UTC m=+34.165825838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls") pod "image-registry-5c96f7fd56-v4sbh" (UID: "865c063b-707c-4239-b057-6194b27e3929") : secret "image-registry-tls" not found
Sep 12 01:45:28.139158 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.139138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:28.139298 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.139240    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 01:45:28.139712 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.139497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/417f469c-1b81-454a-b5a1-bdd3a402ddbc-tmp\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.139848 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.139811    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:28.639693781 +0000 UTC m=+34.166594842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : secret "router-metrics-certs-default" not found
Sep 12 01:45:28.139954 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.139854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/417f469c-1b81-454a-b5a1-bdd3a402ddbc-snapshots\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.140017 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.139963    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-registry-certificates\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.140202 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.140153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/417f469c-1b81-454a-b5a1-bdd3a402ddbc-trusted-ca-bundle\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.141791 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.141770    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-default-certificate\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.141791 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.141786    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-image-registry-private-configuration\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.142312 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.142289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/417f469c-1b81-454a-b5a1-bdd3a402ddbc-serving-cert\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.142402 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.142379    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-stats-auth\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.149720 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.149676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-installation-pull-secrets\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.151438 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.151396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vzj9\" (UniqueName: \"kubernetes.io/projected/59059329-6a69-4148-8545-54cb110614d6-kube-api-access-9vzj9\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.151744 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.151718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr92n\" (UniqueName: \"kubernetes.io/projected/417f469c-1b81-454a-b5a1-bdd3a402ddbc-kube-api-access-pr92n\") pod \"insights-operator-5844b856db-9fnqn\" (UID: \"417f469c-1b81-454a-b5a1-bdd3a402ddbc\") " pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.152373 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.152352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj47h\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-kube-api-access-cj47h\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.152500 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.152391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9fhk\" (UniqueName: \"kubernetes.io/projected/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-kube-api-access-f9fhk\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:28.153028 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.153004    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-bound-sa-token\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.221745 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.221717    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-9fnqn"
Sep 12 01:45:28.239022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.238997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4ndp\" (UniqueName: \"kubernetes.io/projected/5f3f62d7-b8eb-40cb-be5f-f4f4f856e391-kube-api-access-d4ndp\") pod \"ingress-canary-cqnvn\" (UID: \"5f3f62d7-b8eb-40cb-be5f-f4f4f856e391\") " pod="openshift-ingress-canary/ingress-canary-cqnvn"
Sep 12 01:45:28.239128 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprdg\" (UniqueName: \"kubernetes.io/projected/bf43f636-07c2-4d7a-99a3-1307fa989e36-kube-api-access-fprdg\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:28.239128 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239069    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7271188a-6d22-42bb-acd9-f98490da02c8-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-j2m4q\" (UID: \"7271188a-6d22-42bb-acd9-f98490da02c8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.239263 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7271188a-6d22-42bb-acd9-f98490da02c8-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-j2m4q\" (UID: \"7271188a-6d22-42bb-acd9-f98490da02c8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.239263 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6nr7\" (UniqueName: \"kubernetes.io/projected/96d615cb-9d90-4ed9-b005-93bbc0f60f94-kube-api-access-s6nr7\") pod \"volume-data-source-validator-86695b7cc7-hg7mj\" (UID: \"96d615cb-9d90-4ed9-b005-93bbc0f60f94\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj"
Sep 12 01:45:28.239263 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7042ae8-6999-4ac4-beb3-392bb871dc4c-trusted-ca\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.239424 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rcjl\" (UniqueName: \"kubernetes.io/projected/3c454255-f0df-4d2b-8297-39a6f73122d9-kube-api-access-4rcjl\") pod \"network-check-source-6f8b768455-x49c5\" (UID: \"3c454255-f0df-4d2b-8297-39a6f73122d9\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-x49c5"
Sep 12 01:45:28.239424 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ab6ae3e-2127-401a-aef3-e84336a9cefe-serving-cert\") pod \"service-ca-operator-cf7c776b7-qtdkj\" (UID: \"1ab6ae3e-2127-401a-aef3-e84336a9cefe\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.239424 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b83e5ba-8551-45d0-806a-e8c406686503-tmp-dir\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.239424 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:28.239424 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0b83e5ba-8551-45d0-806a-e8c406686503-config-volume\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.239424 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.239706 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5b6k\" (UniqueName: \"kubernetes.io/projected/7271188a-6d22-42bb-acd9-f98490da02c8-kube-api-access-r5b6k\") pod \"kube-storage-version-migrator-operator-6c8f84f897-j2m4q\" (UID: \"7271188a-6d22-42bb-acd9-f98490da02c8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.239706 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ab6ae3e-2127-401a-aef3-e84336a9cefe-config\") pod \"service-ca-operator-cf7c776b7-qtdkj\" (UID: \"1ab6ae3e-2127-401a-aef3-e84336a9cefe\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.239706 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.239536    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 12 01:45:28.239706 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8drz2\" (UniqueName: \"kubernetes.io/projected/1ab6ae3e-2127-401a-aef3-e84336a9cefe-kube-api-access-8drz2\") pod \"service-ca-operator-cf7c776b7-qtdkj\" (UID: \"1ab6ae3e-2127-401a-aef3-e84336a9cefe\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.239706 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.239597    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls podName:bf43f636-07c2-4d7a-99a3-1307fa989e36 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:28.739580666 +0000 UTC m=+34.266481750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-nhmt4" (UID: "bf43f636-07c2-4d7a-99a3-1307fa989e36") : secret "samples-operator-tls" not found
Sep 12 01:45:28.239706 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7271188a-6d22-42bb-acd9-f98490da02c8-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-j2m4q\" (UID: \"7271188a-6d22-42bb-acd9-f98490da02c8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.240007 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7042ae8-6999-4ac4-beb3-392bb871dc4c-config\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.240007 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f3f62d7-b8eb-40cb-be5f-f4f4f856e391-cert\") pod \"ingress-canary-cqnvn\" (UID: \"5f3f62d7-b8eb-40cb-be5f-f4f4f856e391\") " pod="openshift-ingress-canary/ingress-canary-cqnvn"
Sep 12 01:45:28.240007 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7042ae8-6999-4ac4-beb3-392bb871dc4c-serving-cert\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.240007 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxlp\" (UniqueName: \"kubernetes.io/projected/a7042ae8-6999-4ac4-beb3-392bb871dc4c-kube-api-access-rmxlp\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.240007 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvpxv\" (UniqueName: \"kubernetes.io/projected/0b83e5ba-8551-45d0-806a-e8c406686503-kube-api-access-rvpxv\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.240007 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.239999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ab6ae3e-2127-401a-aef3-e84336a9cefe-config\") pod \"service-ca-operator-cf7c776b7-qtdkj\" (UID: \"1ab6ae3e-2127-401a-aef3-e84336a9cefe\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.240646 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.240620    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7042ae8-6999-4ac4-beb3-392bb871dc4c-config\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.240792 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.240768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7042ae8-6999-4ac4-beb3-392bb871dc4c-trusted-ca\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.241757 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.241738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ab6ae3e-2127-401a-aef3-e84336a9cefe-serving-cert\") pod \"service-ca-operator-cf7c776b7-qtdkj\" (UID: \"1ab6ae3e-2127-401a-aef3-e84336a9cefe\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.241757 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.241752    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7271188a-6d22-42bb-acd9-f98490da02c8-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-j2m4q\" (UID: \"7271188a-6d22-42bb-acd9-f98490da02c8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.242346 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.242308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7042ae8-6999-4ac4-beb3-392bb871dc4c-serving-cert\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.251970 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.251945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5b6k\" (UniqueName: \"kubernetes.io/projected/7271188a-6d22-42bb-acd9-f98490da02c8-kube-api-access-r5b6k\") pod \"kube-storage-version-migrator-operator-6c8f84f897-j2m4q\" (UID: \"7271188a-6d22-42bb-acd9-f98490da02c8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.252842 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.252818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fprdg\" (UniqueName: \"kubernetes.io/projected/bf43f636-07c2-4d7a-99a3-1307fa989e36-kube-api-access-fprdg\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:28.253055 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.253023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmxlp\" (UniqueName: \"kubernetes.io/projected/a7042ae8-6999-4ac4-beb3-392bb871dc4c-kube-api-access-rmxlp\") pod \"console-operator-696884fb9c-m6nx8\" (UID: \"a7042ae8-6999-4ac4-beb3-392bb871dc4c\") " pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.253475 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.253431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6nr7\" (UniqueName: \"kubernetes.io/projected/96d615cb-9d90-4ed9-b005-93bbc0f60f94-kube-api-access-s6nr7\") pod \"volume-data-source-validator-86695b7cc7-hg7mj\" (UID: \"96d615cb-9d90-4ed9-b005-93bbc0f60f94\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj"
Sep 12 01:45:28.254088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.254012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8drz2\" (UniqueName: \"kubernetes.io/projected/1ab6ae3e-2127-401a-aef3-e84336a9cefe-kube-api-access-8drz2\") pod \"service-ca-operator-cf7c776b7-qtdkj\" (UID: \"1ab6ae3e-2127-401a-aef3-e84336a9cefe\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.277911 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.277641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj"
Sep 12 01:45:28.328003 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.327924    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:28.340328 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.339708    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"
Sep 12 01:45:28.341012 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.340967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rcjl\" (UniqueName: \"kubernetes.io/projected/3c454255-f0df-4d2b-8297-39a6f73122d9-kube-api-access-4rcjl\") pod \"network-check-source-6f8b768455-x49c5\" (UID: \"3c454255-f0df-4d2b-8297-39a6f73122d9\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-x49c5"
Sep 12 01:45:28.341119 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.341098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b83e5ba-8551-45d0-806a-e8c406686503-tmp-dir\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.341634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.341178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0b83e5ba-8551-45d0-806a-e8c406686503-config-volume\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.341634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.341223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.341634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.341277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f3f62d7-b8eb-40cb-be5f-f4f4f856e391-cert\") pod \"ingress-canary-cqnvn\" (UID: \"5f3f62d7-b8eb-40cb-be5f-f4f4f856e391\") " pod="openshift-ingress-canary/ingress-canary-cqnvn"
Sep 12 01:45:28.341634 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.341398    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 01:45:28.341634 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.341451    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls podName:0b83e5ba-8551-45d0-806a-e8c406686503 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:28.841433314 +0000 UTC m=+34.368334396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls") pod "dns-default-lxb7h" (UID: "0b83e5ba-8551-45d0-806a-e8c406686503") : secret "dns-default-metrics-tls" not found
Sep 12 01:45:28.341634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.341515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvpxv\" (UniqueName: \"kubernetes.io/projected/0b83e5ba-8551-45d0-806a-e8c406686503-kube-api-access-rvpxv\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.341634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.341559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4ndp\" (UniqueName: \"kubernetes.io/projected/5f3f62d7-b8eb-40cb-be5f-f4f4f856e391-kube-api-access-d4ndp\") pod \"ingress-canary-cqnvn\" (UID: \"5f3f62d7-b8eb-40cb-be5f-f4f4f856e391\") " pod="openshift-ingress-canary/ingress-canary-cqnvn"
Sep 12 01:45:28.342047 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.341899    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0b83e5ba-8551-45d0-806a-e8c406686503-config-volume\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.342173 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.342092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b83e5ba-8551-45d0-806a-e8c406686503-tmp-dir\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.349067 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.349019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f3f62d7-b8eb-40cb-be5f-f4f4f856e391-cert\") pod \"ingress-canary-cqnvn\" (UID: \"5f3f62d7-b8eb-40cb-be5f-f4f4f856e391\") " pod="openshift-ingress-canary/ingress-canary-cqnvn"
Sep 12 01:45:28.351295 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.351273    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rcjl\" (UniqueName: \"kubernetes.io/projected/3c454255-f0df-4d2b-8297-39a6f73122d9-kube-api-access-4rcjl\") pod \"network-check-source-6f8b768455-x49c5\" (UID: \"3c454255-f0df-4d2b-8297-39a6f73122d9\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-x49c5"
Sep 12 01:45:28.354045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.354021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4ndp\" (UniqueName: \"kubernetes.io/projected/5f3f62d7-b8eb-40cb-be5f-f4f4f856e391-kube-api-access-d4ndp\") pod \"ingress-canary-cqnvn\" (UID: \"5f3f62d7-b8eb-40cb-be5f-f4f4f856e391\") " pod="openshift-ingress-canary/ingress-canary-cqnvn"
Sep 12 01:45:28.356671 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.356647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvpxv\" (UniqueName: \"kubernetes.io/projected/0b83e5ba-8551-45d0-806a-e8c406686503-kube-api-access-rvpxv\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.368264 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.368238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"
Sep 12 01:45:28.395652 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.395010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x49c5"
Sep 12 01:45:28.414774 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.414535    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cqnvn"
Sep 12 01:45:28.537741 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.537707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-9fnqn"]
Sep 12 01:45:28.541288 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:45:28.541261    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417f469c_1b81_454a_b5a1_bdd3a402ddbc.slice/crio-4f3e50da62b9e3de4a257b9dcc337dce6ae62ecb0e5acd63f6d5cdce4a27fc70 WatchSource:0}: Error finding container 4f3e50da62b9e3de4a257b9dcc337dce6ae62ecb0e5acd63f6d5cdce4a27fc70: Status 404 returned error can't find the container with id 4f3e50da62b9e3de4a257b9dcc337dce6ae62ecb0e5acd63f6d5cdce4a27fc70
Sep 12 01:45:28.567320 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.567299    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj"]
Sep 12 01:45:28.569833 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:45:28.569809    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d615cb_9d90_4ed9_b005_93bbc0f60f94.slice/crio-d0ac8a6578e6ef9b1214a64c317df3ee17db27eb28c93d6bc5526c5ab6332209 WatchSource:0}: Error finding container d0ac8a6578e6ef9b1214a64c317df3ee17db27eb28c93d6bc5526c5ab6332209: Status 404 returned error can't find the container with id d0ac8a6578e6ef9b1214a64c317df3ee17db27eb28c93d6bc5526c5ab6332209
Sep 12 01:45:28.625122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.625092    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q"]
Sep 12 01:45:28.628495 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.628412    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-m6nx8"]
Sep 12 01:45:28.629515 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:45:28.629486    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7271188a_6d22_42bb_acd9_f98490da02c8.slice/crio-d3de652f91e14f09d940c62d3b4203ff2ca9335d0668a05a38ae8f7764b46cf7 WatchSource:0}: Error finding container d3de652f91e14f09d940c62d3b4203ff2ca9335d0668a05a38ae8f7764b46cf7: Status 404 returned error can't find the container with id d3de652f91e14f09d940c62d3b4203ff2ca9335d0668a05a38ae8f7764b46cf7
Sep 12 01:45:28.630606 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:45:28.630577    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7042ae8_6999_4ac4_beb3_392bb871dc4c.slice/crio-e90c3f5080250cdd049200c7bb67730bd0a85c8cb5f28c8e0cd2e0e0d0d20945 WatchSource:0}: Error finding container e90c3f5080250cdd049200c7bb67730bd0a85c8cb5f28c8e0cd2e0e0d0d20945: Status 404 returned error can't find the container with id e90c3f5080250cdd049200c7bb67730bd0a85c8cb5f28c8e0cd2e0e0d0d20945
Sep 12 01:45:28.644484 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.644461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.644499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.644554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.644620    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.644642    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c96f7fd56-v4sbh: secret "image-registry-tls" not found
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.644692    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls podName:865c063b-707c-4239-b057-6194b27e3929 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:29.644673519 +0000 UTC m=+35.171574592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls") pod "image-registry-5c96f7fd56-v4sbh" (UID: "865c063b-707c-4239-b057-6194b27e3929") : secret "image-registry-tls" not found
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.644698    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.644710    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:29.644700895 +0000 UTC m=+35.171601959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : configmap references non-existent config key: service-ca.crt
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.644741    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:29.644724139 +0000 UTC m=+35.171625210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : secret "router-metrics-certs-default" not found
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.644776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.644870    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:28.645031 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.644899    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls podName:2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:29.644887887 +0000 UTC m=+35.171789022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhx4j" (UID: "2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96") : secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:28.674196 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.674095    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj"]
Sep 12 01:45:28.677902 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:45:28.677881    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ab6ae3e_2127_401a_aef3_e84336a9cefe.slice/crio-33687a4e3194f447c5bf60d6336fcc160bd9ad1dee32e69343990770359d9237 WatchSource:0}: Error finding container 33687a4e3194f447c5bf60d6336fcc160bd9ad1dee32e69343990770359d9237: Status 404 returned error can't find the container with id 33687a4e3194f447c5bf60d6336fcc160bd9ad1dee32e69343990770359d9237
Sep 12 01:45:28.693848 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.693797    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-x49c5"]
Sep 12 01:45:28.696017 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:45:28.695988    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c454255_f0df_4d2b_8297_39a6f73122d9.slice/crio-e83bca3506dda20f2578f44fae22885ce4d8047bf6ea97edd65594142697d550 WatchSource:0}: Error finding container e83bca3506dda20f2578f44fae22885ce4d8047bf6ea97edd65594142697d550: Status 404 returned error can't find the container with id e83bca3506dda20f2578f44fae22885ce4d8047bf6ea97edd65594142697d550
Sep 12 01:45:28.715529 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.715500    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cqnvn"]
Sep 12 01:45:28.717986 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:45:28.717958    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f3f62d7_b8eb_40cb_be5f_f4f4f856e391.slice/crio-b0b50f2c546f533116a56261251faf43f712061e5cced4fed1b17bd828b5936b WatchSource:0}: Error finding container b0b50f2c546f533116a56261251faf43f712061e5cced4fed1b17bd828b5936b: Status 404 returned error can't find the container with id b0b50f2c546f533116a56261251faf43f712061e5cced4fed1b17bd828b5936b
Sep 12 01:45:28.745950 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.745922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:28.746096 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.746070    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 12 01:45:28.746197 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.746141    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls podName:bf43f636-07c2-4d7a-99a3-1307fa989e36 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:29.746124691 +0000 UTC m=+35.273025753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-nhmt4" (UID: "bf43f636-07c2-4d7a-99a3-1307fa989e36") : secret "samples-operator-tls" not found
Sep 12 01:45:28.846923 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.846887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:28.847076 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.847054    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 01:45:28.847138 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:28.847121    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls podName:0b83e5ba-8551-45d0-806a-e8c406686503 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:29.847105794 +0000 UTC m=+35.374006852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls") pod "dns-default-lxb7h" (UID: "0b83e5ba-8551-45d0-806a-e8c406686503") : secret "dns-default-metrics-tls" not found
Sep 12 01:45:28.973390 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.973314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:28.973531 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.973314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:28.977004 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.976943    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hmzff\""
Sep 12 01:45:28.977135 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.977052    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 12 01:45:28.978155 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:28.977206    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fqtlc\""
Sep 12 01:45:29.272160 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.271589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj" event={"ID":"1ab6ae3e-2127-401a-aef3-e84336a9cefe","Type":"ContainerStarted","Data":"33687a4e3194f447c5bf60d6336fcc160bd9ad1dee32e69343990770359d9237"}
Sep 12 01:45:29.273139 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.273109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8" event={"ID":"a7042ae8-6999-4ac4-beb3-392bb871dc4c","Type":"ContainerStarted","Data":"e90c3f5080250cdd049200c7bb67730bd0a85c8cb5f28c8e0cd2e0e0d0d20945"}
Sep 12 01:45:29.276317 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.276292    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cqnvn" event={"ID":"5f3f62d7-b8eb-40cb-be5f-f4f4f856e391","Type":"ContainerStarted","Data":"b0b50f2c546f533116a56261251faf43f712061e5cced4fed1b17bd828b5936b"}
Sep 12 01:45:29.277904 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.277874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q" event={"ID":"7271188a-6d22-42bb-acd9-f98490da02c8","Type":"ContainerStarted","Data":"d3de652f91e14f09d940c62d3b4203ff2ca9335d0668a05a38ae8f7764b46cf7"}
Sep 12 01:45:29.280854 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.280788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x49c5" event={"ID":"3c454255-f0df-4d2b-8297-39a6f73122d9","Type":"ContainerStarted","Data":"e83bca3506dda20f2578f44fae22885ce4d8047bf6ea97edd65594142697d550"}
Sep 12 01:45:29.282836 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.282778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj" event={"ID":"96d615cb-9d90-4ed9-b005-93bbc0f60f94","Type":"ContainerStarted","Data":"d0ac8a6578e6ef9b1214a64c317df3ee17db27eb28c93d6bc5526c5ab6332209"}
Sep 12 01:45:29.285639 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.285582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9fnqn" event={"ID":"417f469c-1b81-454a-b5a1-bdd3a402ddbc","Type":"ContainerStarted","Data":"4f3e50da62b9e3de4a257b9dcc337dce6ae62ecb0e5acd63f6d5cdce4a27fc70"}
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.653443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.653559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.653592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.653648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.653823    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.653888    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:31.653869499 +0000 UTC m=+37.180770573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : secret "router-metrics-certs-default" not found
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.654704    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.654727    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c96f7fd56-v4sbh: secret "image-registry-tls" not found
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.654777    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls podName:865c063b-707c-4239-b057-6194b27e3929 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:31.654763069 +0000 UTC m=+37.181664140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls") pod "image-registry-5c96f7fd56-v4sbh" (UID: "865c063b-707c-4239-b057-6194b27e3929") : secret "image-registry-tls" not found
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.654844    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.654885    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls podName:2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:31.654872908 +0000 UTC m=+37.181773971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhx4j" (UID: "2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96") : secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:29.654974 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.654961    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:31.654951107 +0000 UTC m=+37.181852179 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : configmap references non-existent config key: service-ca.crt
Sep 12 01:45:29.755679 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.755040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:29.755679 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.755276    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 12 01:45:29.755679 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.755338    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls podName:bf43f636-07c2-4d7a-99a3-1307fa989e36 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:31.755319624 +0000 UTC m=+37.282220704 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-nhmt4" (UID: "bf43f636-07c2-4d7a-99a3-1307fa989e36") : secret "samples-operator-tls" not found
Sep 12 01:45:29.856930 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:29.856322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:29.856930 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.856535    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 01:45:29.856930 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:29.856593    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls podName:0b83e5ba-8551-45d0-806a-e8c406686503 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:31.856574842 +0000 UTC m=+37.383475921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls") pod "dns-default-lxb7h" (UID: "0b83e5ba-8551-45d0-806a-e8c406686503") : secret "dns-default-metrics-tls" not found
Sep 12 01:45:31.676617 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:31.676378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:31.676617 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:31.676436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:31.676617 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:31.676495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:31.676617 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.676564    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 01:45:31.676617 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.676588    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c96f7fd56-v4sbh: secret "image-registry-tls" not found
Sep 12 01:45:31.676617 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.676616    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:35.676593922 +0000 UTC m=+41.203494993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : configmap references non-existent config key: service-ca.crt
Sep 12 01:45:31.677582 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.676660    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 01:45:31.677582 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:31.676715    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:31.677582 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.676762    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:35.67674155 +0000 UTC m=+41.203642624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : secret "router-metrics-certs-default" not found
Sep 12 01:45:31.677582 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.676782    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:31.677582 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.676807    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls podName:865c063b-707c-4239-b057-6194b27e3929 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:35.676795274 +0000 UTC m=+41.203696338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls") pod "image-registry-5c96f7fd56-v4sbh" (UID: "865c063b-707c-4239-b057-6194b27e3929") : secret "image-registry-tls" not found
Sep 12 01:45:31.677582 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.676833    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls podName:2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:35.676823288 +0000 UTC m=+41.203724354 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhx4j" (UID: "2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96") : secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:31.777829 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:31.777794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:31.778001 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.777946    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 12 01:45:31.778051 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.778043    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls podName:bf43f636-07c2-4d7a-99a3-1307fa989e36 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:35.778027957 +0000 UTC m=+41.304929015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-nhmt4" (UID: "bf43f636-07c2-4d7a-99a3-1307fa989e36") : secret "samples-operator-tls" not found
Sep 12 01:45:31.878527 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:31.878482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:31.878720 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.878635    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 01:45:31.878720 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:31.878705    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls podName:0b83e5ba-8551-45d0-806a-e8c406686503 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:35.878689716 +0000 UTC m=+41.405590775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls") pod "dns-default-lxb7h" (UID: "0b83e5ba-8551-45d0-806a-e8c406686503") : secret "dns-default-metrics-tls" not found
Sep 12 01:45:35.709521 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:35.709480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:35.709567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:35.709661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:35.709696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.709657    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.709752    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.709820    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls podName:2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:43.709798795 +0000 UTC m=+49.236699871 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhx4j" (UID: "2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96") : secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.709843    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.709860    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:43.709849148 +0000 UTC m=+49.236750206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : secret "router-metrics-certs-default" not found
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.709863    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c96f7fd56-v4sbh: secret "image-registry-tls" not found
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.709878    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:43.709870325 +0000 UTC m=+49.236771385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : configmap references non-existent config key: service-ca.crt
Sep 12 01:45:35.709937 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.709920    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls podName:865c063b-707c-4239-b057-6194b27e3929 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:43.709901032 +0000 UTC m=+49.236802123 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls") pod "image-registry-5c96f7fd56-v4sbh" (UID: "865c063b-707c-4239-b057-6194b27e3929") : secret "image-registry-tls" not found
Sep 12 01:45:35.811099 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:35.811063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:35.811286 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.811196    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 12 01:45:35.811286 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.811272    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls podName:bf43f636-07c2-4d7a-99a3-1307fa989e36 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:43.811257844 +0000 UTC m=+49.338158907 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-nhmt4" (UID: "bf43f636-07c2-4d7a-99a3-1307fa989e36") : secret "samples-operator-tls" not found
Sep 12 01:45:35.912505 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:35.912421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:35.912659 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.912592    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 01:45:35.912718 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:35.912664    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls podName:0b83e5ba-8551-45d0-806a-e8c406686503 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:43.912644577 +0000 UTC m=+49.439545649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls") pod "dns-default-lxb7h" (UID: "0b83e5ba-8551-45d0-806a-e8c406686503") : secret "dns-default-metrics-tls" not found
Sep 12 01:45:38.322037 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.321941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj" event={"ID":"1ab6ae3e-2127-401a-aef3-e84336a9cefe","Type":"ContainerStarted","Data":"3144fb3666d447b702514b4c1c2326f184708e11c2b7f56563435879fe1c5e45"}
Sep 12 01:45:38.324114 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.324089    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/0.log"
Sep 12 01:45:38.324262 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.324131    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7042ae8-6999-4ac4-beb3-392bb871dc4c" containerID="4c64edef84d59ac8ab423332a94272c61e92caf645dbd61babd50c87fc69bef8" exitCode=255
Sep 12 01:45:38.324262 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.324200    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8" event={"ID":"a7042ae8-6999-4ac4-beb3-392bb871dc4c","Type":"ContainerDied","Data":"4c64edef84d59ac8ab423332a94272c61e92caf645dbd61babd50c87fc69bef8"}
Sep 12 01:45:38.324589 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.324421    2580 scope.go:117] "RemoveContainer" containerID="4c64edef84d59ac8ab423332a94272c61e92caf645dbd61babd50c87fc69bef8"
Sep 12 01:45:38.328226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.328028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cqnvn" event={"ID":"5f3f62d7-b8eb-40cb-be5f-f4f4f856e391","Type":"ContainerStarted","Data":"bf6efceaaa07b014be3ee4c28cbf39fd4e6571b8b9bb930383d3565d095a4b06"}
Sep 12 01:45:38.328226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.328089    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:38.328226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.328173    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:38.330806 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.330778    2580 generic.go:356] "Generic (PLEG): container finished" podID="c50a1b1c-0c8c-48b5-bed5-96f6b3591444" containerID="28cd778aadee61241e7696e106a36395a1815a18f1bdae06da5a25b2d68cf9a0" exitCode=0
Sep 12 01:45:38.330900 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.330865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl2v2" event={"ID":"c50a1b1c-0c8c-48b5-bed5-96f6b3591444","Type":"ContainerDied","Data":"28cd778aadee61241e7696e106a36395a1815a18f1bdae06da5a25b2d68cf9a0"}
Sep 12 01:45:38.335745 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.335722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q" event={"ID":"7271188a-6d22-42bb-acd9-f98490da02c8","Type":"ContainerStarted","Data":"0ac7b5eef4f0befee56bd6e94c83d11bf684bf21ba87a1cc77beb1497177c849"}
Sep 12 01:45:38.338184 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.338161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x49c5" event={"ID":"3c454255-f0df-4d2b-8297-39a6f73122d9","Type":"ContainerStarted","Data":"93623883671b9bf047cbafe825fe40718585cefe473b70a944989f5b39ed3abb"}
Sep 12 01:45:38.341386 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.340647    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj" event={"ID":"96d615cb-9d90-4ed9-b005-93bbc0f60f94","Type":"ContainerStarted","Data":"897ba3bd92dc8b093c3748de741a934a900de992aec6ec203e4045e213af6d01"}
Sep 12 01:45:38.341978 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.341744    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj" podStartSLOduration=19.669046794 podStartE2EDuration="28.3417273s" totalImagesPullingTime="8.672680506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:38.341168236 +0000 UTC m=+43.868069317" watchObservedRunningTime="2026-09-12 01:45:38.3417273 +0000 UTC m=+43.868628381"
Sep 12 01:45:38.342847 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.342820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9fnqn" event={"ID":"417f469c-1b81-454a-b5a1-bdd3a402ddbc","Type":"ContainerStarted","Data":"81f7690ddc01264bff7ed7345d7c0eabf9673b6cb199cf533e8b0cd626d256e3"}
Sep 12 01:45:38.360922 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.360180    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x49c5" podStartSLOduration=21.70483057 podStartE2EDuration="30.360160184s" totalImagesPullingTime="8.655329614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:38.359596247 +0000 UTC m=+43.886497328" watchObservedRunningTime="2026-09-12 01:45:38.360160184 +0000 UTC m=+43.887061259"
Sep 12 01:45:38.413524 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.413465    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q" podStartSLOduration=22.692124093 podStartE2EDuration="31.413425399s" totalImagesPullingTime="8.721301306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:38.412011596 +0000 UTC m=+43.938912684" watchObservedRunningTime="2026-09-12 01:45:38.413425399 +0000 UTC m=+43.940326480"
Sep 12 01:45:38.459977 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.459865    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cqnvn" podStartSLOduration=2.826435405 podStartE2EDuration="11.459843589s" totalImagesPullingTime="8.633408184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:38.432057548 +0000 UTC m=+43.958958626" watchObservedRunningTime="2026-09-12 01:45:38.459843589 +0000 UTC m=+43.986744672"
Sep 12 01:45:38.480997 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.480836    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-9fnqn" podStartSLOduration=29.671687866 podStartE2EDuration="38.480815201s" totalImagesPullingTime="8.809127335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:38.461308553 +0000 UTC m=+43.988209633" watchObservedRunningTime="2026-09-12 01:45:38.480815201 +0000 UTC m=+44.007716282"
Sep 12 01:45:38.482203 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.481586    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-hg7mj" podStartSLOduration=27.700362873 podStartE2EDuration="36.481574334s" totalImagesPullingTime="8.781211461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:38.480921956 +0000 UTC m=+44.007823037" watchObservedRunningTime="2026-09-12 01:45:38.481574334 +0000 UTC m=+44.008475416"
Sep 12 01:45:38.770009 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.769974    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hqh4b"]
Sep 12 01:45:38.783837 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.783800    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hqh4b"]
Sep 12 01:45:38.783982 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.783955    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.786419 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.786393    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 12 01:45:38.786632 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.786617    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 12 01:45:38.786849 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.786834    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9fddj\""
Sep 12 01:45:38.838105 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.838042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8888f357-81fc-4037-abd4-b9c18d605cc4-data-volume\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.838248 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.838158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.838248 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.838225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8888f357-81fc-4037-abd4-b9c18d605cc4-crio-socket\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.838359 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.838295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkt4r\" (UniqueName: \"kubernetes.io/projected/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-api-access-nkt4r\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.838359 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.838338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.939474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.939556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8888f357-81fc-4037-abd4-b9c18d605cc4-data-volume\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.939646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.939704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8888f357-81fc-4037-abd4-b9c18d605cc4-crio-socket\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.939774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkt4r\" (UniqueName: \"kubernetes.io/projected/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-api-access-nkt4r\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:38.940196    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:38.940310    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls podName:8888f357-81fc-4037-abd4-b9c18d605cc4 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:39.440273386 +0000 UTC m=+44.967174459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqh4b" (UID: "8888f357-81fc-4037-abd4-b9c18d605cc4") : secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.940733    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8888f357-81fc-4037-abd4-b9c18d605cc4-crio-socket\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:38.940814    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.940814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8888f357-81fc-4037-abd4-b9c18d605cc4-data-volume\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:38.940867 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:38.940863    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm podName:8888f357-81fc-4037-abd4-b9c18d605cc4 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:39.440846154 +0000 UTC m=+44.967747217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hqh4b" (UID: "8888f357-81fc-4037-abd4-b9c18d605cc4") : configmap "kube-rbac-proxy" not found
Sep 12 01:45:38.949714 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:38.949659    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkt4r\" (UniqueName: \"kubernetes.io/projected/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-api-access-nkt4r\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:39.352591 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.352557    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 01:45:39.353097 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.353078    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/0.log"
Sep 12 01:45:39.353176 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.353121    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7042ae8-6999-4ac4-beb3-392bb871dc4c" containerID="6c52ec2bb820074ccf3a6e9c5e886090541652cf329780995b25a5d133ee6edb" exitCode=255
Sep 12 01:45:39.353264 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.353234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8" event={"ID":"a7042ae8-6999-4ac4-beb3-392bb871dc4c","Type":"ContainerDied","Data":"6c52ec2bb820074ccf3a6e9c5e886090541652cf329780995b25a5d133ee6edb"}
Sep 12 01:45:39.353377 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.353360    2580 scope.go:117] "RemoveContainer" containerID="4c64edef84d59ac8ab423332a94272c61e92caf645dbd61babd50c87fc69bef8"
Sep 12 01:45:39.353627 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.353605    2580 scope.go:117] "RemoveContainer" containerID="6c52ec2bb820074ccf3a6e9c5e886090541652cf329780995b25a5d133ee6edb"
Sep 12 01:45:39.353903 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:39.353872    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-m6nx8_openshift-console-operator(a7042ae8-6999-4ac4-beb3-392bb871dc4c)\"" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8" podUID="a7042ae8-6999-4ac4-beb3-392bb871dc4c"
Sep 12 01:45:39.357389 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.357368    2580 generic.go:356] "Generic (PLEG): container finished" podID="c50a1b1c-0c8c-48b5-bed5-96f6b3591444" containerID="379105863bf370ae8fa82f979440ed48486314fff913864b405f38b1d2074d9f" exitCode=0
Sep 12 01:45:39.357490 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.357463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl2v2" event={"ID":"c50a1b1c-0c8c-48b5-bed5-96f6b3591444","Type":"ContainerDied","Data":"379105863bf370ae8fa82f979440ed48486314fff913864b405f38b1d2074d9f"}
Sep 12 01:45:39.444813 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.444620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:39.444813 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:39.444708    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 01:45:39.444813 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:39.444783    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm podName:8888f357-81fc-4037-abd4-b9c18d605cc4 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:40.444761521 +0000 UTC m=+45.971662599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hqh4b" (UID: "8888f357-81fc-4037-abd4-b9c18d605cc4") : configmap "kube-rbac-proxy" not found
Sep 12 01:45:39.445617 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:39.445205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:39.445617 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:39.445595    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:39.445757 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:39.445653    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls podName:8888f357-81fc-4037-abd4-b9c18d605cc4 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:40.445638378 +0000 UTC m=+45.972539441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqh4b" (UID: "8888f357-81fc-4037-abd4-b9c18d605cc4") : secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:40.363034 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:40.363008    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 01:45:40.363549 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:40.363357    2580 scope.go:117] "RemoveContainer" containerID="6c52ec2bb820074ccf3a6e9c5e886090541652cf329780995b25a5d133ee6edb"
Sep 12 01:45:40.363549 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:40.363531    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-m6nx8_openshift-console-operator(a7042ae8-6999-4ac4-beb3-392bb871dc4c)\"" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8" podUID="a7042ae8-6999-4ac4-beb3-392bb871dc4c"
Sep 12 01:45:40.366775 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:40.366746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xl2v2" event={"ID":"c50a1b1c-0c8c-48b5-bed5-96f6b3591444","Type":"ContainerStarted","Data":"048d0f4467d00add30eda9f3ad3e9d2bedaadc136d51090769fae0ca38d3304f"}
Sep 12 01:45:40.399291 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:40.399245    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xl2v2" podStartSLOduration=8.124756675 podStartE2EDuration="45.399230924s" totalImagesPullingTime="37.274474249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:40.398579963 +0000 UTC m=+45.925481045" watchObservedRunningTime="2026-09-12 01:45:40.399230924 +0000 UTC m=+45.926131995"
Sep 12 01:45:40.453563 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:40.453527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:40.453760 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:40.453606    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 01:45:40.453760 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:40.453665    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm podName:8888f357-81fc-4037-abd4-b9c18d605cc4 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:42.453647783 +0000 UTC m=+47.980548850 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hqh4b" (UID: "8888f357-81fc-4037-abd4-b9c18d605cc4") : configmap "kube-rbac-proxy" not found
Sep 12 01:45:40.453760 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:40.453731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:40.453956 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:40.453916    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:40.454006 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:40.453970    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls podName:8888f357-81fc-4037-abd4-b9c18d605cc4 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:42.453953506 +0000 UTC m=+47.980854582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqh4b" (UID: "8888f357-81fc-4037-abd4-b9c18d605cc4") : secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:40.544567 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:40.544541    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-svxhl_890d8019-49a3-4b97-b6d9-ff60814cb6af/dns-node-resolver/0.log"
Sep 12 01:45:41.547481 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:41.547455    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zjw29_0e050df8-40a2-410e-a291-3177dabbb8f8/node-ca/0.log"
Sep 12 01:45:42.144815 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:42.144783    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cqnvn_5f3f62d7-b8eb-40cb-be5f-f4f4f856e391/serve-healthcheck-canary/0.log"
Sep 12 01:45:42.471664 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:42.471588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:42.471803 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:42.471707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:42.471803 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:42.471746    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:42.471876 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:42.471814    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls podName:8888f357-81fc-4037-abd4-b9c18d605cc4 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:46.471795245 +0000 UTC m=+51.998696304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqh4b" (UID: "8888f357-81fc-4037-abd4-b9c18d605cc4") : secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:42.471876 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:42.471821    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 01:45:42.471876 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:42.471872    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm podName:8888f357-81fc-4037-abd4-b9c18d605cc4 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:46.471858441 +0000 UTC m=+51.998759499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hqh4b" (UID: "8888f357-81fc-4037-abd4-b9c18d605cc4") : configmap "kube-rbac-proxy" not found
Sep 12 01:45:43.783191 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:43.783161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:43.783256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:43.783284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.783312    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.783375    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls podName:2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:59.783360971 +0000 UTC m=+65.310262057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-mhx4j" (UID: "2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96") : secret "cluster-monitoring-operator-tls" not found
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.783395    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:59.783384715 +0000 UTC m=+65.310285773 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : configmap references non-existent config key: service-ca.crt
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.783395    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:43.783318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.783411    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c96f7fd56-v4sbh: secret "image-registry-tls" not found
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.783444    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.783515    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls podName:865c063b-707c-4239-b057-6194b27e3929 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:59.783506156 +0000 UTC m=+65.310407214 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls") pod "image-registry-5c96f7fd56-v4sbh" (UID: "865c063b-707c-4239-b057-6194b27e3929") : secret "image-registry-tls" not found
Sep 12 01:45:43.783561 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.783544    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs podName:59059329-6a69-4148-8545-54cb110614d6 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:59.783532996 +0000 UTC m=+65.310434069 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs") pod "router-default-76f959c4b4-5lb49" (UID: "59059329-6a69-4148-8545-54cb110614d6") : secret "router-metrics-certs-default" not found
Sep 12 01:45:43.884836 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:43.884810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:43.884962 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.884919    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 12 01:45:43.885000 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.884962    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls podName:bf43f636-07c2-4d7a-99a3-1307fa989e36 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:59.88495151 +0000 UTC m=+65.411852568 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-nhmt4" (UID: "bf43f636-07c2-4d7a-99a3-1307fa989e36") : secret "samples-operator-tls" not found
Sep 12 01:45:43.985316 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:43.985280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:45:43.985449 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.985410    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 01:45:43.985496 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:43.985460    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls podName:0b83e5ba-8551-45d0-806a-e8c406686503 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:59.985446696 +0000 UTC m=+65.512347754 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls") pod "dns-default-lxb7h" (UID: "0b83e5ba-8551-45d0-806a-e8c406686503") : secret "dns-default-metrics-tls" not found
Sep 12 01:45:46.508184 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:46.508146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:46.508587 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:46.508311    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:46.508587 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:46.508367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:46.508587 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:46.508383    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls podName:8888f357-81fc-4037-abd4-b9c18d605cc4 nodeName:}" failed. No retries permitted until 2026-09-12 01:45:54.508364814 +0000 UTC m=+60.035265891 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqh4b" (UID: "8888f357-81fc-4037-abd4-b9c18d605cc4") : secret "insights-runtime-extractor-tls" not found
Sep 12 01:45:46.508835 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:46.508819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8888f357-81fc-4037-abd4-b9c18d605cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:48.328724 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:48.328694    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:48.329081 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:48.328744    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:45:48.329081 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:48.329039    2580 scope.go:117] "RemoveContainer" containerID="6c52ec2bb820074ccf3a6e9c5e886090541652cf329780995b25a5d133ee6edb"
Sep 12 01:45:48.329191 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:45:48.329173    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-m6nx8_openshift-console-operator(a7042ae8-6999-4ac4-beb3-392bb871dc4c)\"" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8" podUID="a7042ae8-6999-4ac4-beb3-392bb871dc4c"
Sep 12 01:45:52.257166 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:52.257137    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gmpjd"
Sep 12 01:45:54.580264 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:54.580233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:54.582438 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:54.582418    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8888f357-81fc-4037-abd4-b9c18d605cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqh4b\" (UID: \"8888f357-81fc-4037-abd4-b9c18d605cc4\") " pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:54.695124 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:54.695075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hqh4b"
Sep 12 01:45:54.976303 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:45:54.976273    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8888f357_81fc_4037_abd4_b9c18d605cc4.slice/crio-0023445f9f2c28b25eb79a8c648ce66021abee6fcaeb2fa985264e7d3c94d63d WatchSource:0}: Error finding container 0023445f9f2c28b25eb79a8c648ce66021abee6fcaeb2fa985264e7d3c94d63d: Status 404 returned error can't find the container with id 0023445f9f2c28b25eb79a8c648ce66021abee6fcaeb2fa985264e7d3c94d63d
Sep 12 01:45:54.977938 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:54.977913    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hqh4b"]
Sep 12 01:45:55.413336 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:55.413298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqh4b" event={"ID":"8888f357-81fc-4037-abd4-b9c18d605cc4","Type":"ContainerStarted","Data":"fbd80ceeb4163966c670cfc59a178150269b6d1214838ec4d19848807426fd36"}
Sep 12 01:45:55.413492 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:55.413369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqh4b" event={"ID":"8888f357-81fc-4037-abd4-b9c18d605cc4","Type":"ContainerStarted","Data":"0023445f9f2c28b25eb79a8c648ce66021abee6fcaeb2fa985264e7d3c94d63d"}
Sep 12 01:45:56.422584 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:56.422543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqh4b" event={"ID":"8888f357-81fc-4037-abd4-b9c18d605cc4","Type":"ContainerStarted","Data":"ec12b4e64a389d96da00f409de866cf68cd77385ed2ffe76b2e2fe772040ecdd"}
Sep 12 01:45:57.427855 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:57.427775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqh4b" event={"ID":"8888f357-81fc-4037-abd4-b9c18d605cc4","Type":"ContainerStarted","Data":"928bbc76c8aac8974f6ac0909cbe826dae34565cf28189290bfad3cb389f45a7"}
Sep 12 01:45:57.444966 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:57.444920    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hqh4b" podStartSLOduration=17.438411266 podStartE2EDuration="19.444905469s" totalImagesPullingTime="2.006494203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:45:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:45:57.443920739 +0000 UTC m=+62.970821818" watchObservedRunningTime="2026-09-12 01:45:57.444905469 +0000 UTC m=+62.971806594"
Sep 12 01:45:59.724621 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.724575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:59.725094 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.724634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:59.725094 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.724707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:59.727417 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.727391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4458x\" (UniqueName: \"kubernetes.io/projected/b3a8d7a4-4a7f-474a-8e27-c9a2a4595835-kube-api-access-4458x\") pod \"network-check-target-75ddj\" (UID: \"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835\") " pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:59.727853 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.727835    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 12 01:45:59.728755 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.728739    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 12 01:45:59.737631 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.737610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b590634-a38d-4e33-931d-b7a784f3d07d-original-pull-secret\") pod \"global-pull-secret-syncer-2cbp5\" (UID: \"7b590634-a38d-4e33-931d-b7a784f3d07d\") " pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:59.737716 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.737703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ae26889-09cb-4e31-9af1-4d560513d4af-metrics-certs\") pod \"network-metrics-daemon-w2745\" (UID: \"9ae26889-09cb-4e31-9af1-4d560513d4af\") " pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:59.823502 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.823475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2cbp5"
Sep 12 01:45:59.825795 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.825772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:59.825900 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.825815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:59.825900 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.825855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:59.826005 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.825905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:59.826467 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.826448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59059329-6a69-4148-8545-54cb110614d6-service-ca-bundle\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:59.828300 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.828271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59059329-6a69-4148-8545-54cb110614d6-metrics-certs\") pod \"router-default-76f959c4b4-5lb49\" (UID: \"59059329-6a69-4148-8545-54cb110614d6\") " pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:45:59.828659 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.828638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-mhx4j\" (UID: \"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:45:59.828744 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.828725    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"image-registry-5c96f7fd56-v4sbh\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") " pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:45:59.891126 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.891100    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fqtlc\""
Sep 12 01:45:59.896367 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.896343    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hmzff\""
Sep 12 01:45:59.899150 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.899134    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2745"
Sep 12 01:45:59.904943 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.904918    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:45:59.926515 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.926486    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:45:59.928733 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:45:59.928704    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf43f636-07c2-4d7a-99a3-1307fa989e36-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-nhmt4\" (UID: \"bf43f636-07c2-4d7a-99a3-1307fa989e36\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:46:00.007438 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.007413    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2s9js\""
Sep 12 01:46:00.015378 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.015363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:46:00.027023 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.026999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:46:00.029231 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.029187    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b83e5ba-8551-45d0-806a-e8c406686503-metrics-tls\") pod \"dns-default-lxb7h\" (UID: \"0b83e5ba-8551-45d0-806a-e8c406686503\") " pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:46:00.047739 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.047708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5jwvz\""
Sep 12 01:46:00.056016 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.055995    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"
Sep 12 01:46:00.063163 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.063142    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-txvrz\""
Sep 12 01:46:00.071107 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.070894    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:46:00.092866 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.092798    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2cbp5"]
Sep 12 01:46:00.096472 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:00.096436    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b590634_a38d_4e33_931d_b7a784f3d07d.slice/crio-3fc8e64fd194405cc62f13870157ad3d960f6a73929971b0113f4d6434efcb9f WatchSource:0}: Error finding container 3fc8e64fd194405cc62f13870157ad3d960f6a73929971b0113f4d6434efcb9f: Status 404 returned error can't find the container with id 3fc8e64fd194405cc62f13870157ad3d960f6a73929971b0113f4d6434efcb9f
Sep 12 01:46:00.102081 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.101924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vwpzs\""
Sep 12 01:46:00.110972 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.110656    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"
Sep 12 01:46:00.174799 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.174724    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w2745"]
Sep 12 01:46:00.179204 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:00.179157    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae26889_09cb_4e31_9af1_4d560513d4af.slice/crio-dacb047eaad0c4818683751dce74f6821e1c7567bfa1d9b14fa8dca44c1ec918 WatchSource:0}: Error finding container dacb047eaad0c4818683751dce74f6821e1c7567bfa1d9b14fa8dca44c1ec918: Status 404 returned error can't find the container with id dacb047eaad0c4818683751dce74f6821e1c7567bfa1d9b14fa8dca44c1ec918
Sep 12 01:46:00.180009 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:00.179972    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a8d7a4_4a7f_474a_8e27_c9a2a4595835.slice/crio-ca94a14ac704ad6f7af85eead1ed3910b045a4c88e4831c8e638064321826a7f WatchSource:0}: Error finding container ca94a14ac704ad6f7af85eead1ed3910b045a4c88e4831c8e638064321826a7f: Status 404 returned error can't find the container with id ca94a14ac704ad6f7af85eead1ed3910b045a4c88e4831c8e638064321826a7f
Sep 12 01:46:00.183712 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.183694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-75ddj"]
Sep 12 01:46:00.242353 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.242333    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gmqgn\""
Sep 12 01:46:00.250466 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.250425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:46:00.282895 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.282862    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c96f7fd56-v4sbh"]
Sep 12 01:46:00.285720 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:00.285665    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod865c063b_707c_4239_b057_6194b27e3929.slice/crio-1fa06ec43738641dec27a8d5410713e813e66e2b0cfd471d5dafb27329dd69c9 WatchSource:0}: Error finding container 1fa06ec43738641dec27a8d5410713e813e66e2b0cfd471d5dafb27329dd69c9: Status 404 returned error can't find the container with id 1fa06ec43738641dec27a8d5410713e813e66e2b0cfd471d5dafb27329dd69c9
Sep 12 01:46:00.333913 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.333882    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j"]
Sep 12 01:46:00.339170 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:00.339142    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f22d7ed_e1a6_465d_9ac5_c11f6b1a3d96.slice/crio-46ba75dbbbde5607976d5449310bb4f3099f8a9eeecb4a0b732ecbd0be47b3d1 WatchSource:0}: Error finding container 46ba75dbbbde5607976d5449310bb4f3099f8a9eeecb4a0b732ecbd0be47b3d1: Status 404 returned error can't find the container with id 46ba75dbbbde5607976d5449310bb4f3099f8a9eeecb4a0b732ecbd0be47b3d1
Sep 12 01:46:00.346027 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.345959    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76f959c4b4-5lb49"]
Sep 12 01:46:00.348179 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:00.348157    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59059329_6a69_4148_8545_54cb110614d6.slice/crio-a964247d6081c86fa6957737222cfffc5b39c72ab4c84042c092c02c9d7b519d WatchSource:0}: Error finding container a964247d6081c86fa6957737222cfffc5b39c72ab4c84042c092c02c9d7b519d: Status 404 returned error can't find the container with id a964247d6081c86fa6957737222cfffc5b39c72ab4c84042c092c02c9d7b519d
Sep 12 01:46:00.387947 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.387922    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4"]
Sep 12 01:46:00.440312 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.440264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4" event={"ID":"bf43f636-07c2-4d7a-99a3-1307fa989e36","Type":"ContainerStarted","Data":"d29ca6b2353873f179308ab889b98231018ded0a194ecb9d6c2923d60df31252"}
Sep 12 01:46:00.442154 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.442122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76f959c4b4-5lb49" event={"ID":"59059329-6a69-4148-8545-54cb110614d6","Type":"ContainerStarted","Data":"cdf84cbc80f340c4e686971a8758491fd3c39884e6debde6da64d5edaf14bf45"}
Sep 12 01:46:00.442287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.442181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76f959c4b4-5lb49" event={"ID":"59059329-6a69-4148-8545-54cb110614d6","Type":"ContainerStarted","Data":"a964247d6081c86fa6957737222cfffc5b39c72ab4c84042c092c02c9d7b519d"}
Sep 12 01:46:00.443690 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.443660    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j" event={"ID":"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96","Type":"ContainerStarted","Data":"46ba75dbbbde5607976d5449310bb4f3099f8a9eeecb4a0b732ecbd0be47b3d1"}
Sep 12 01:46:00.445395 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.445369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh" event={"ID":"865c063b-707c-4239-b057-6194b27e3929","Type":"ContainerStarted","Data":"84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47"}
Sep 12 01:46:00.445508 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.445420    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:46:00.445508 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.445443    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh" event={"ID":"865c063b-707c-4239-b057-6194b27e3929","Type":"ContainerStarted","Data":"1fa06ec43738641dec27a8d5410713e813e66e2b0cfd471d5dafb27329dd69c9"}
Sep 12 01:46:00.446658 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.446632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w2745" event={"ID":"9ae26889-09cb-4e31-9af1-4d560513d4af","Type":"ContainerStarted","Data":"dacb047eaad0c4818683751dce74f6821e1c7567bfa1d9b14fa8dca44c1ec918"}
Sep 12 01:46:00.448186 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.448162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-75ddj" event={"ID":"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835","Type":"ContainerStarted","Data":"e19ef0a5c7de67a1a74c7a236fac483413a711d9bd6c0b29839a4e67c1fefc4b"}
Sep 12 01:46:00.448289 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.448224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-75ddj" event={"ID":"b3a8d7a4-4a7f-474a-8e27-c9a2a4595835","Type":"ContainerStarted","Data":"ca94a14ac704ad6f7af85eead1ed3910b045a4c88e4831c8e638064321826a7f"}
Sep 12 01:46:00.448289 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.448265    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:46:00.449339 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.449320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2cbp5" event={"ID":"7b590634-a38d-4e33-931d-b7a784f3d07d","Type":"ContainerStarted","Data":"3fc8e64fd194405cc62f13870157ad3d960f6a73929971b0113f4d6434efcb9f"}
Sep 12 01:46:00.462787 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.462743    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-76f959c4b4-5lb49" podStartSLOduration=60.462730016 podStartE2EDuration="1m0.462730016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:00.462706254 +0000 UTC m=+65.989607372" watchObservedRunningTime="2026-09-12 01:46:00.462730016 +0000 UTC m=+65.989631111"
Sep 12 01:46:00.479869 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.479825    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh" podStartSLOduration=65.479812126 podStartE2EDuration="1m5.479812126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:00.479586681 +0000 UTC m=+66.006487772" watchObservedRunningTime="2026-09-12 01:46:00.479812126 +0000 UTC m=+66.006713206"
Sep 12 01:46:00.498426 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.498375    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-75ddj" podStartSLOduration=65.498356701 podStartE2EDuration="1m5.498356701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:00.496921711 +0000 UTC m=+66.023822792" watchObservedRunningTime="2026-09-12 01:46:00.498356701 +0000 UTC m=+66.025257784"
Sep 12 01:46:00.517732 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:00.517699    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lxb7h"]
Sep 12 01:46:00.521343 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:00.521315    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b83e5ba_8551_45d0_806a_e8c406686503.slice/crio-d510b40b99067370b874bf69814a201572ecd8468698bc4788b8b07eee177df5 WatchSource:0}: Error finding container d510b40b99067370b874bf69814a201572ecd8468698bc4788b8b07eee177df5: Status 404 returned error can't find the container with id d510b40b99067370b874bf69814a201572ecd8468698bc4788b8b07eee177df5
Sep 12 01:46:01.072714 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:01.072673    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:46:01.076073 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:01.075814    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:46:01.458681 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:01.458601    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lxb7h" event={"ID":"0b83e5ba-8551-45d0-806a-e8c406686503","Type":"ContainerStarted","Data":"d510b40b99067370b874bf69814a201572ecd8468698bc4788b8b07eee177df5"}
Sep 12 01:46:01.459941 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:01.459528    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:46:01.460758 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:01.460735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-76f959c4b4-5lb49"
Sep 12 01:46:02.465699 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:02.465604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w2745" event={"ID":"9ae26889-09cb-4e31-9af1-4d560513d4af","Type":"ContainerStarted","Data":"972373724d675d2bc6140ab8622c0776a4060528e214503c83b47edabe1d50c0"}
Sep 12 01:46:02.465699 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:02.465663    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w2745" event={"ID":"9ae26889-09cb-4e31-9af1-4d560513d4af","Type":"ContainerStarted","Data":"9619d26a11d8d4422436ce0a6e8ff71c938e50c3c56e46412cada5425bc88bb3"}
Sep 12 01:46:02.484232 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:02.484166    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w2745" podStartSLOduration=66.149450935 podStartE2EDuration="1m7.484148471s" totalImagesPullingTime="1.334697536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:02.48304669 +0000 UTC m=+68.009947770" watchObservedRunningTime="2026-09-12 01:46:02.484148471 +0000 UTC m=+68.011049555"
Sep 12 01:46:02.973560 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:02.973529    2580 scope.go:117] "RemoveContainer" containerID="6c52ec2bb820074ccf3a6e9c5e886090541652cf329780995b25a5d133ee6edb"
Sep 12 01:46:05.485388 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.485340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4" event={"ID":"bf43f636-07c2-4d7a-99a3-1307fa989e36","Type":"ContainerStarted","Data":"0a5509529ed5312a8caf569d5411446130ef668a5bb06c5bc2761ed6a1bfb087"}
Sep 12 01:46:05.485818 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.485420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4" event={"ID":"bf43f636-07c2-4d7a-99a3-1307fa989e36","Type":"ContainerStarted","Data":"5246a7841148dcf35ed8b4a41891262ad95330033fdba627d25a4640cfe76d8b"}
Sep 12 01:46:05.487900 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.487868    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 01:46:05.488030 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.487985    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8" event={"ID":"a7042ae8-6999-4ac4-beb3-392bb871dc4c","Type":"ContainerStarted","Data":"4338182b3ff717b93f15646bd9801861e7f94f7ee4de45344f4c25cf284b8398"}
Sep 12 01:46:05.488316 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.488302    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:46:05.490828 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.490808    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j" event={"ID":"2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96","Type":"ContainerStarted","Data":"caa8c942c5bdb9fda0a145eab8b20fe9bdb76d545ad4201d084751e07996e767"}
Sep 12 01:46:05.495385 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.495365    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8"
Sep 12 01:46:05.495539 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.495523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2cbp5" event={"ID":"7b590634-a38d-4e33-931d-b7a784f3d07d","Type":"ContainerStarted","Data":"5a9a97be0e04d164a6fa0ca18bf11928cb02ae9439a51403d6cad4ebed45363a"}
Sep 12 01:46:05.499421 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.497405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lxb7h" event={"ID":"0b83e5ba-8551-45d0-806a-e8c406686503","Type":"ContainerStarted","Data":"43bff5dea21a8a3ec3fb5d12d2252bdb126494a2d162d89ea3891b01430bf60e"}
Sep 12 01:46:05.499421 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.497440    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lxb7h" event={"ID":"0b83e5ba-8551-45d0-806a-e8c406686503","Type":"ContainerStarted","Data":"6d89fcf8d90989d84e86e9e56dad4205a2f88e9d00011c8afa7b5200e2dc5b1c"}
Sep 12 01:46:05.499421 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.497790    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:46:05.506309 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.505922    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-nhmt4" podStartSLOduration=58.962389215 podStartE2EDuration="1m3.505906159s" totalImagesPullingTime="4.543516944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:05.505019844 +0000 UTC m=+71.031920926" watchObservedRunningTime="2026-09-12 01:46:05.505906159 +0000 UTC m=+71.032807239"
Sep 12 01:46:05.532562 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.532502    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lxb7h" podStartSLOduration=34.100143955 podStartE2EDuration="38.532482298s" totalImagesPullingTime="4.432338343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:05.530862019 +0000 UTC m=+71.057763111" watchObservedRunningTime="2026-09-12 01:46:05.532482298 +0000 UTC m=+71.059383379"
Sep 12 01:46:05.546690 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.546639    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2cbp5" podStartSLOduration=65.369667985 podStartE2EDuration="1m10.546620118s" totalImagesPullingTime="5.176952133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:05.546171221 +0000 UTC m=+71.073072306" watchObservedRunningTime="2026-09-12 01:46:05.546620118 +0000 UTC m=+71.073521200"
Sep 12 01:46:05.564787 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.564734    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-mhx4j" podStartSLOduration=60.954950828 podStartE2EDuration="1m5.564689001s" totalImagesPullingTime="4.609738173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:05.562785796 +0000 UTC m=+71.089686875" watchObservedRunningTime="2026-09-12 01:46:05.564689001 +0000 UTC m=+71.091590084"
Sep 12 01:46:05.587231 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:05.587166    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-m6nx8" podStartSLOduration=53.430546894 podStartE2EDuration="1m2.587152752s" totalImagesPullingTime="9.156605858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:45:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:05.587027536 +0000 UTC m=+71.113928620" watchObservedRunningTime="2026-09-12 01:46:05.587152752 +0000 UTC m=+71.114053872"
Sep 12 01:46:14.123591 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.123553    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78ddfddc-8h6dt"]
Sep 12 01:46:14.127367 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.127339    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.129927 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.129903    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 12 01:46:14.129927 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.129921    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5x28t\""
Sep 12 01:46:14.130122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.129947    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 12 01:46:14.130979 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.130957    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 12 01:46:14.131153 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.131132    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 12 01:46:14.131269 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.130994    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 12 01:46:14.131269 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.130999    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 12 01:46:14.131269 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.131051    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 12 01:46:14.136081 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.136053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78ddfddc-8h6dt"]
Sep 12 01:46:14.243574 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.243529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-oauth-serving-cert\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.243574 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.243578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-serving-cert\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.243811 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.243598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt8f9\" (UniqueName: \"kubernetes.io/projected/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-kube-api-access-rt8f9\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.243811 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.243671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-oauth-config\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.243811 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.243719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-config\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.243811 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.243785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-service-ca\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.344779 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.344729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-oauth-config\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.344779 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.344785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-config\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.345015 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.344832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-service-ca\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.345015 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.344865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-oauth-serving-cert\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.345015 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.344900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-serving-cert\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.345148 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.345121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt8f9\" (UniqueName: \"kubernetes.io/projected/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-kube-api-access-rt8f9\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.345792 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.345757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-service-ca\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.345792 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.345779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-oauth-serving-cert\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.345982 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.345766    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-config\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.347449 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.347413    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-oauth-config\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.347574 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.347505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-serving-cert\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.353328 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.353286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt8f9\" (UniqueName: \"kubernetes.io/projected/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-kube-api-access-rt8f9\") pod \"console-78ddfddc-8h6dt\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") " pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.438911 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.438822    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:14.713741 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:14.713713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78ddfddc-8h6dt"]
Sep 12 01:46:14.716258 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:14.716201    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bf3f5b9_cebb_46cb_a7a1_8783c44aac95.slice/crio-2286751c049eead9c5cebd932e3cabb94074542884a6e3b7e243b2cc3e03ef28 WatchSource:0}: Error finding container 2286751c049eead9c5cebd932e3cabb94074542884a6e3b7e243b2cc3e03ef28: Status 404 returned error can't find the container with id 2286751c049eead9c5cebd932e3cabb94074542884a6e3b7e243b2cc3e03ef28
Sep 12 01:46:15.538320 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:15.538277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78ddfddc-8h6dt" event={"ID":"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95","Type":"ContainerStarted","Data":"2286751c049eead9c5cebd932e3cabb94074542884a6e3b7e243b2cc3e03ef28"}
Sep 12 01:46:16.509154 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.509127    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lxb7h"
Sep 12 01:46:16.916256 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.916135    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"]
Sep 12 01:46:16.919137 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.919112    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:16.921711 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.921651    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 12 01:46:16.921711 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.921684    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gjrkh\""
Sep 12 01:46:16.921930 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.921717    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 12 01:46:16.921930 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.921782    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 12 01:46:16.927380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.927348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"]
Sep 12 01:46:16.958448 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.958416    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hcpqd"]
Sep 12 01:46:16.967070 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.967042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:16.969681 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.969653    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 12 01:46:16.969842 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.969771    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 12 01:46:16.970005 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.969986    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 12 01:46:16.970163 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:16.970145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xdtzd\""
Sep 12 01:46:17.070924 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.070880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ee9079f-e83a-4b51-b108-c8f1d3064f52-sys\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.071125 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.070934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ee9079f-e83a-4b51-b108-c8f1d3064f52-metrics-client-ca\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.071125 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.070971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-textfile\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.071125 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071003    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-wtmp\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.071323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/832c6f15-f6ad-411e-b33b-43987485def4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.071323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4shq\" (UniqueName: \"kubernetes.io/projected/832c6f15-f6ad-411e-b33b-43987485def4-kube-api-access-l4shq\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.071323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071258    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.071323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1ee9079f-e83a-4b51-b108-c8f1d3064f52-root\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.071498 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-accelerators-collector-config\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.071498 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-tls\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.071498 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z8xq\" (UniqueName: \"kubernetes.io/projected/1ee9079f-e83a-4b51-b108-c8f1d3064f52-kube-api-access-5z8xq\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.071498 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/832c6f15-f6ad-411e-b33b-43987485def4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.071688 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.071504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/832c6f15-f6ad-411e-b33b-43987485def4-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.172122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/832c6f15-f6ad-411e-b33b-43987485def4-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.172122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ee9079f-e83a-4b51-b108-c8f1d3064f52-sys\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ee9079f-e83a-4b51-b108-c8f1d3064f52-metrics-client-ca\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172434 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-textfile\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172434 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-wtmp\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172434 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/832c6f15-f6ad-411e-b33b-43987485def4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.172434 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ee9079f-e83a-4b51-b108-c8f1d3064f52-sys\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172434 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4shq\" (UniqueName: \"kubernetes.io/projected/832c6f15-f6ad-411e-b33b-43987485def4-kube-api-access-l4shq\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.172434 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172434 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1ee9079f-e83a-4b51-b108-c8f1d3064f52-root\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172434 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-accelerators-collector-config\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172834 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-tls\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172834 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z8xq\" (UniqueName: \"kubernetes.io/projected/1ee9079f-e83a-4b51-b108-c8f1d3064f52-kube-api-access-5z8xq\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172834 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/832c6f15-f6ad-411e-b33b-43987485def4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.172834 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-textfile\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172834 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172665    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-wtmp\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.172834 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ee9079f-e83a-4b51-b108-c8f1d3064f52-metrics-client-ca\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.173122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172889    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/832c6f15-f6ad-411e-b33b-43987485def4-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.173122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.172943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1ee9079f-e83a-4b51-b108-c8f1d3064f52-root\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.173122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.173066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-accelerators-collector-config\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.175707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.175666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/832c6f15-f6ad-411e-b33b-43987485def4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.175853 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.175787    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-tls\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.175853 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.175837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/832c6f15-f6ad-411e-b33b-43987485def4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.176323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.176298    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ee9079f-e83a-4b51-b108-c8f1d3064f52-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.180768 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.180737    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4shq\" (UniqueName: \"kubernetes.io/projected/832c6f15-f6ad-411e-b33b-43987485def4-kube-api-access-l4shq\") pod \"openshift-state-metrics-67dcf8c979-49nx7\" (UID: \"832c6f15-f6ad-411e-b33b-43987485def4\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.180902 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.180808    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z8xq\" (UniqueName: \"kubernetes.io/projected/1ee9079f-e83a-4b51-b108-c8f1d3064f52-kube-api-access-5z8xq\") pod \"node-exporter-hcpqd\" (UID: \"1ee9079f-e83a-4b51-b108-c8f1d3064f52\") " pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.232417 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.232374    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"
Sep 12 01:46:17.279484 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:17.279450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hcpqd"
Sep 12 01:46:17.780436 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:17.780403    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee9079f_e83a_4b51_b108_c8f1d3064f52.slice/crio-589f5dda226d97488f156a744267e643ee9cdba320a145af5176bf9526e4af82 WatchSource:0}: Error finding container 589f5dda226d97488f156a744267e643ee9cdba320a145af5176bf9526e4af82: Status 404 returned error can't find the container with id 589f5dda226d97488f156a744267e643ee9cdba320a145af5176bf9526e4af82
Sep 12 01:46:18.050154 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:18.050053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7"]
Sep 12 01:46:18.053097 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:18.053067    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod832c6f15_f6ad_411e_b33b_43987485def4.slice/crio-298ad8751bc71099b2f66dce1343ec112255afd4c7ca302975a4d06a6fb16248 WatchSource:0}: Error finding container 298ad8751bc71099b2f66dce1343ec112255afd4c7ca302975a4d06a6fb16248: Status 404 returned error can't find the container with id 298ad8751bc71099b2f66dce1343ec112255afd4c7ca302975a4d06a6fb16248
Sep 12 01:46:18.555080 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:18.554994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78ddfddc-8h6dt" event={"ID":"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95","Type":"ContainerStarted","Data":"44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997"}
Sep 12 01:46:18.557980 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:18.557937    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7" event={"ID":"832c6f15-f6ad-411e-b33b-43987485def4","Type":"ContainerStarted","Data":"4b77aa220cec4fc318983ea697aa55b8b226c00e7e0a332096ec6d1710af7cf9"}
Sep 12 01:46:18.558133 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:18.558021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7" event={"ID":"832c6f15-f6ad-411e-b33b-43987485def4","Type":"ContainerStarted","Data":"4bddf8afa769d48e6533c41dde7fe9a9e259de08b28d40140e93b01ea8a76985"}
Sep 12 01:46:18.558133 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:18.558059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7" event={"ID":"832c6f15-f6ad-411e-b33b-43987485def4","Type":"ContainerStarted","Data":"298ad8751bc71099b2f66dce1343ec112255afd4c7ca302975a4d06a6fb16248"}
Sep 12 01:46:18.559474 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:18.559447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcpqd" event={"ID":"1ee9079f-e83a-4b51-b108-c8f1d3064f52","Type":"ContainerStarted","Data":"589f5dda226d97488f156a744267e643ee9cdba320a145af5176bf9526e4af82"}
Sep 12 01:46:18.576091 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:18.576025    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78ddfddc-8h6dt" podStartSLOduration=1.481177834 podStartE2EDuration="4.575999139s" totalImagesPullingTime="3.094821305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:18.574627596 +0000 UTC m=+84.101528678" watchObservedRunningTime="2026-09-12 01:46:18.575999139 +0000 UTC m=+84.102900219"
Sep 12 01:46:19.564791 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:19.564755    2580 generic.go:356] "Generic (PLEG): container finished" podID="1ee9079f-e83a-4b51-b108-c8f1d3064f52" containerID="e15df3bd463b4c7c9b03b4ccbd8f33e42202e6e9dd8953386cc392bae9bc27af" exitCode=0
Sep 12 01:46:19.565179 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:19.564841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcpqd" event={"ID":"1ee9079f-e83a-4b51-b108-c8f1d3064f52","Type":"ContainerDied","Data":"e15df3bd463b4c7c9b03b4ccbd8f33e42202e6e9dd8953386cc392bae9bc27af"}
Sep 12 01:46:19.567184 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:19.567151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7" event={"ID":"832c6f15-f6ad-411e-b33b-43987485def4","Type":"ContainerStarted","Data":"c1c5e0f485f97f9dd010cab7868d4a669e48cb75091ad0360dfc8bba5a54195a"}
Sep 12 01:46:19.599003 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:19.598946    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-49nx7" podStartSLOduration=2.635158633 podStartE2EDuration="3.598928829s" totalImagesPullingTime="963.770196ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:46:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:19.597388854 +0000 UTC m=+85.124289934" watchObservedRunningTime="2026-09-12 01:46:19.598928829 +0000 UTC m=+85.125829908"
Sep 12 01:46:20.573082 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:20.573038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcpqd" event={"ID":"1ee9079f-e83a-4b51-b108-c8f1d3064f52","Type":"ContainerStarted","Data":"2ce11e9d46325f16fc10fc6c48ef1f8046d7fe41f25c052802a7f0f6b2eb2bab"}
Sep 12 01:46:20.573627 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:20.573140    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcpqd" event={"ID":"1ee9079f-e83a-4b51-b108-c8f1d3064f52","Type":"ContainerStarted","Data":"12607cef8336d880279aa18f1e6e5d629676964030634cee99299da9e21d7dcd"}
Sep 12 01:46:20.592894 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:20.592837    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hcpqd" podStartSLOduration=3.656299443 podStartE2EDuration="4.592819237s" totalImagesPullingTime="857.765654ms" totalInitContainerRuntime="78.75414ms" isStatefulPod=true podCreationTimestamp="2026-09-12 01:46:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:20.591500721 +0000 UTC m=+86.118401811" watchObservedRunningTime="2026-09-12 01:46:20.592819237 +0000 UTC m=+86.119720317"
Sep 12 01:46:21.464111 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:21.464081    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:46:24.047277 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:24.047236    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c96f7fd56-v4sbh"]
Sep 12 01:46:24.439282 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:24.439159    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:24.439465 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:24.439291    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:24.444103 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:24.444074    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:24.592509 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:24.592469    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:46:26.158069 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.158036    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-785c467696-jw6cr"]
Sep 12 01:46:26.161354 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.161332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.168836 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.168811    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 12 01:46:26.172505 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.172482    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-785c467696-jw6cr"]
Sep 12 01:46:26.251865 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.251834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-service-ca\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.251865 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.251865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-trusted-ca-bundle\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.252060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.251884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrqb2\" (UniqueName: \"kubernetes.io/projected/c3554fb6-98fa-4280-b755-851379fe9c79-kube-api-access-vrqb2\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.252060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.251907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-oauth-serving-cert\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.252060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.251985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-serving-cert\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.252060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.252018    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-console-config\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.252060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.252044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-oauth-config\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.353162 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.353125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-console-config\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.353162 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.353165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-oauth-config\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.353403 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.353191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-service-ca\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.353403 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.353230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-trusted-ca-bundle\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.353403 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.353359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrqb2\" (UniqueName: \"kubernetes.io/projected/c3554fb6-98fa-4280-b755-851379fe9c79-kube-api-access-vrqb2\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.353557 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.353430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-oauth-serving-cert\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.353557 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.353502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-serving-cert\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.353977 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.353953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-console-config\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.354095 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.354077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-trusted-ca-bundle\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.354095 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.354023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-oauth-serving-cert\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.354178 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.353997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-service-ca\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.355645 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.355617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-oauth-config\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.355803 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.355785    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-serving-cert\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.360947 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.360923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrqb2\" (UniqueName: \"kubernetes.io/projected/c3554fb6-98fa-4280-b755-851379fe9c79-kube-api-access-vrqb2\") pod \"console-785c467696-jw6cr\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") " pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.470193 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.470111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:26.740143 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:26.740076    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-785c467696-jw6cr"]
Sep 12 01:46:26.743645 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:46:26.743615    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3554fb6_98fa_4280_b755_851379fe9c79.slice/crio-181b07496fe56b31e211a655f784bfc1504407d35779f3cc19cfe20c5872029d WatchSource:0}: Error finding container 181b07496fe56b31e211a655f784bfc1504407d35779f3cc19cfe20c5872029d: Status 404 returned error can't find the container with id 181b07496fe56b31e211a655f784bfc1504407d35779f3cc19cfe20c5872029d
Sep 12 01:46:27.602272 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:27.602238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-785c467696-jw6cr" event={"ID":"c3554fb6-98fa-4280-b755-851379fe9c79","Type":"ContainerStarted","Data":"6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033"}
Sep 12 01:46:27.602621 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:27.602290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-785c467696-jw6cr" event={"ID":"c3554fb6-98fa-4280-b755-851379fe9c79","Type":"ContainerStarted","Data":"181b07496fe56b31e211a655f784bfc1504407d35779f3cc19cfe20c5872029d"}
Sep 12 01:46:27.623459 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:27.623412    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-785c467696-jw6cr" podStartSLOduration=1.6233933980000002 podStartE2EDuration="1.623393398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:46:27.62119519 +0000 UTC m=+93.148096269" watchObservedRunningTime="2026-09-12 01:46:27.623393398 +0000 UTC m=+93.150294478"
Sep 12 01:46:31.462389 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:31.462316    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-75ddj"
Sep 12 01:46:36.470993 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:36.470957    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:36.471445 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:36.471046    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:36.475755 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:36.475735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:36.640961 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:36.640934    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:46:36.696131 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:36.696099    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78ddfddc-8h6dt"]
Sep 12 01:46:43.665452 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:43.665413    2580 generic.go:356] "Generic (PLEG): container finished" podID="7271188a-6d22-42bb-acd9-f98490da02c8" containerID="0ac7b5eef4f0befee56bd6e94c83d11bf684bf21ba87a1cc77beb1497177c849" exitCode=0
Sep 12 01:46:43.665857 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:43.665495    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q" event={"ID":"7271188a-6d22-42bb-acd9-f98490da02c8","Type":"ContainerDied","Data":"0ac7b5eef4f0befee56bd6e94c83d11bf684bf21ba87a1cc77beb1497177c849"}
Sep 12 01:46:43.665857 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:43.665835    2580 scope.go:117] "RemoveContainer" containerID="0ac7b5eef4f0befee56bd6e94c83d11bf684bf21ba87a1cc77beb1497177c849"
Sep 12 01:46:44.671117 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:44.671087    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-j2m4q" event={"ID":"7271188a-6d22-42bb-acd9-f98490da02c8","Type":"ContainerStarted","Data":"610a3f0ddbbda16c17bdf6a65186d353552fc0f975f20b893a25d23f19ea6678"}
Sep 12 01:46:49.068434 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.068387    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh" podUID="865c063b-707c-4239-b057-6194b27e3929" containerName="registry" containerID="cri-o://84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47" gracePeriod=30
Sep 12 01:46:49.313436 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.313412    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:46:49.452036 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.451953    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-image-registry-private-configuration\") pod \"865c063b-707c-4239-b057-6194b27e3929\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") "
Sep 12 01:46:49.452036 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.451989    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-trusted-ca\") pod \"865c063b-707c-4239-b057-6194b27e3929\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") "
Sep 12 01:46:49.452036 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.452011    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-bound-sa-token\") pod \"865c063b-707c-4239-b057-6194b27e3929\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") "
Sep 12 01:46:49.452327 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.452060    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865c063b-707c-4239-b057-6194b27e3929-ca-trust-extracted\") pod \"865c063b-707c-4239-b057-6194b27e3929\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") "
Sep 12 01:46:49.452327 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.452096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-installation-pull-secrets\") pod \"865c063b-707c-4239-b057-6194b27e3929\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") "
Sep 12 01:46:49.452327 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.452122    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") pod \"865c063b-707c-4239-b057-6194b27e3929\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") "
Sep 12 01:46:49.452327 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.452151    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj47h\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-kube-api-access-cj47h\") pod \"865c063b-707c-4239-b057-6194b27e3929\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") "
Sep 12 01:46:49.452327 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.452187    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-registry-certificates\") pod \"865c063b-707c-4239-b057-6194b27e3929\" (UID: \"865c063b-707c-4239-b057-6194b27e3929\") "
Sep 12 01:46:49.452577 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.452505    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "865c063b-707c-4239-b057-6194b27e3929" (UID: "865c063b-707c-4239-b057-6194b27e3929"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:46:49.452759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.452733    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "865c063b-707c-4239-b057-6194b27e3929" (UID: "865c063b-707c-4239-b057-6194b27e3929"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:46:49.454648 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.454612    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-kube-api-access-cj47h" (OuterVolumeSpecName: "kube-api-access-cj47h") pod "865c063b-707c-4239-b057-6194b27e3929" (UID: "865c063b-707c-4239-b057-6194b27e3929"). InnerVolumeSpecName "kube-api-access-cj47h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:46:49.454771 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.454738    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "865c063b-707c-4239-b057-6194b27e3929" (UID: "865c063b-707c-4239-b057-6194b27e3929"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:46:49.454828 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.454783    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "865c063b-707c-4239-b057-6194b27e3929" (UID: "865c063b-707c-4239-b057-6194b27e3929"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:46:49.455109 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.455084    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "865c063b-707c-4239-b057-6194b27e3929" (UID: "865c063b-707c-4239-b057-6194b27e3929"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:46:49.455178 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.455126    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "865c063b-707c-4239-b057-6194b27e3929" (UID: "865c063b-707c-4239-b057-6194b27e3929"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:46:49.460410 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.460380    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865c063b-707c-4239-b057-6194b27e3929-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "865c063b-707c-4239-b057-6194b27e3929" (UID: "865c063b-707c-4239-b057-6194b27e3929"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:46:49.552856 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.552826    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865c063b-707c-4239-b057-6194b27e3929-ca-trust-extracted\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:46:49.552856 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.552852    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-installation-pull-secrets\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:46:49.553018 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.552867    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-registry-tls\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:46:49.553018 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.552880    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj47h\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-kube-api-access-cj47h\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:46:49.553018 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.552891    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-registry-certificates\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:46:49.553018 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.552905    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865c063b-707c-4239-b057-6194b27e3929-image-registry-private-configuration\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:46:49.553018 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.552919    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865c063b-707c-4239-b057-6194b27e3929-trusted-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:46:49.553018 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.552930    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865c063b-707c-4239-b057-6194b27e3929-bound-sa-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:46:49.691975 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.691943    2580 generic.go:356] "Generic (PLEG): container finished" podID="865c063b-707c-4239-b057-6194b27e3929" containerID="84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47" exitCode=0
Sep 12 01:46:49.692148 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.691991    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh"
Sep 12 01:46:49.692148 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.692031    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh" event={"ID":"865c063b-707c-4239-b057-6194b27e3929","Type":"ContainerDied","Data":"84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47"}
Sep 12 01:46:49.692148 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.692094    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c96f7fd56-v4sbh" event={"ID":"865c063b-707c-4239-b057-6194b27e3929","Type":"ContainerDied","Data":"1fa06ec43738641dec27a8d5410713e813e66e2b0cfd471d5dafb27329dd69c9"}
Sep 12 01:46:49.692148 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.692122    2580 scope.go:117] "RemoveContainer" containerID="84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47"
Sep 12 01:46:49.700614 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.700596    2580 scope.go:117] "RemoveContainer" containerID="84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47"
Sep 12 01:46:49.700884 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:46:49.700864    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47\": container with ID starting with 84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47 not found: ID does not exist" containerID="84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47"
Sep 12 01:46:49.700919 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.700894    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47"} err="failed to get container status \"84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47\": rpc error: code = NotFound desc = could not find container \"84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47\": container with ID starting with 84df8c1b49f5a5a2478ab8922935d7b6feffcfff55d5f556bcb917fe9d311a47 not found: ID does not exist"
Sep 12 01:46:49.717184 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.717134    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c96f7fd56-v4sbh"]
Sep 12 01:46:49.724967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:49.724943    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5c96f7fd56-v4sbh"]
Sep 12 01:46:50.977517 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:50.977484    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="865c063b-707c-4239-b057-6194b27e3929" path="/var/lib/kubelet/pods/865c063b-707c-4239-b057-6194b27e3929/volumes"
Sep 12 01:46:53.710307 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:53.710265    2580 generic.go:356] "Generic (PLEG): container finished" podID="1ab6ae3e-2127-401a-aef3-e84336a9cefe" containerID="3144fb3666d447b702514b4c1c2326f184708e11c2b7f56563435879fe1c5e45" exitCode=0
Sep 12 01:46:53.710685 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:53.710327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj" event={"ID":"1ab6ae3e-2127-401a-aef3-e84336a9cefe","Type":"ContainerDied","Data":"3144fb3666d447b702514b4c1c2326f184708e11c2b7f56563435879fe1c5e45"}
Sep 12 01:46:53.710685 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:53.710649    2580 scope.go:117] "RemoveContainer" containerID="3144fb3666d447b702514b4c1c2326f184708e11c2b7f56563435879fe1c5e45"
Sep 12 01:46:54.716899 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:54.716860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-qtdkj" event={"ID":"1ab6ae3e-2127-401a-aef3-e84336a9cefe","Type":"ContainerStarted","Data":"8fb166660caa10491f84b5c958de07f8332e4893808d20f5717aa64e380f526a"}
Sep 12 01:46:58.734341 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:58.734247    2580 generic.go:356] "Generic (PLEG): container finished" podID="417f469c-1b81-454a-b5a1-bdd3a402ddbc" containerID="81f7690ddc01264bff7ed7345d7c0eabf9673b6cb199cf533e8b0cd626d256e3" exitCode=0
Sep 12 01:46:58.734341 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:58.734324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9fnqn" event={"ID":"417f469c-1b81-454a-b5a1-bdd3a402ddbc","Type":"ContainerDied","Data":"81f7690ddc01264bff7ed7345d7c0eabf9673b6cb199cf533e8b0cd626d256e3"}
Sep 12 01:46:58.734844 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:58.734778    2580 scope.go:117] "RemoveContainer" containerID="81f7690ddc01264bff7ed7345d7c0eabf9673b6cb199cf533e8b0cd626d256e3"
Sep 12 01:46:59.739645 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:46:59.739613    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9fnqn" event={"ID":"417f469c-1b81-454a-b5a1-bdd3a402ddbc","Type":"ContainerStarted","Data":"99134a347abd174c58e077fc1ec09c2abea723292ce37528f08797ef799a10c7"}
Sep 12 01:47:01.717537 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:01.717481    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78ddfddc-8h6dt" podUID="7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" containerName="console" containerID="cri-o://44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997" gracePeriod=15
Sep 12 01:47:01.955131 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:01.955108    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:47:02.053429 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.053402    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt8f9\" (UniqueName: \"kubernetes.io/projected/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-kube-api-access-rt8f9\") pod \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") "
Sep 12 01:47:02.053604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.053440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-serving-cert\") pod \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") "
Sep 12 01:47:02.053604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.053474    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-service-ca\") pod \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") "
Sep 12 01:47:02.053721 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.053598    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-oauth-config\") pod \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") "
Sep 12 01:47:02.053721 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.053629    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-config\") pod \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") "
Sep 12 01:47:02.053721 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.053657    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-oauth-serving-cert\") pod \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\" (UID: \"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95\") "
Sep 12 01:47:02.053884 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.053857    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-service-ca" (OuterVolumeSpecName: "service-ca") pod "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" (UID: "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:47:02.053941 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.053926    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:47:02.054037 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.054009    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-config" (OuterVolumeSpecName: "console-config") pod "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" (UID: "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:47:02.054105 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.054080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" (UID: "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:47:02.055712 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.055689    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-kube-api-access-rt8f9" (OuterVolumeSpecName: "kube-api-access-rt8f9") pod "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" (UID: "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95"). InnerVolumeSpecName "kube-api-access-rt8f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:47:02.055790 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.055730    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" (UID: "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:47:02.055829 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.055785    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" (UID: "7bf3f5b9-cebb-46cb-a7a1-8783c44aac95"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:47:02.155002 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.154975    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt8f9\" (UniqueName: \"kubernetes.io/projected/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-kube-api-access-rt8f9\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:47:02.155002 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.154998    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-serving-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:47:02.155111 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.155008    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-oauth-config\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:47:02.155111 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.155019    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-console-config\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:47:02.155111 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.155027    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95-oauth-serving-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:47:02.752529 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.752492    2580 generic.go:356] "Generic (PLEG): container finished" podID="7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" containerID="44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997" exitCode=0
Sep 12 01:47:02.752903 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.752541    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78ddfddc-8h6dt"
Sep 12 01:47:02.752903 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.752542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78ddfddc-8h6dt" event={"ID":"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95","Type":"ContainerDied","Data":"44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997"}
Sep 12 01:47:02.752903 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.752593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78ddfddc-8h6dt" event={"ID":"7bf3f5b9-cebb-46cb-a7a1-8783c44aac95","Type":"ContainerDied","Data":"2286751c049eead9c5cebd932e3cabb94074542884a6e3b7e243b2cc3e03ef28"}
Sep 12 01:47:02.752903 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.752620    2580 scope.go:117] "RemoveContainer" containerID="44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997"
Sep 12 01:47:02.761246 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.761206    2580 scope.go:117] "RemoveContainer" containerID="44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997"
Sep 12 01:47:02.761554 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:47:02.761531    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997\": container with ID starting with 44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997 not found: ID does not exist" containerID="44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997"
Sep 12 01:47:02.761626 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.761565    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997"} err="failed to get container status \"44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997\": rpc error: code = NotFound desc = could not find container \"44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997\": container with ID starting with 44b22144bc671644bba264f7a3f7a8fe5d7aa92c15030de829a30e540e5ce997 not found: ID does not exist"
Sep 12 01:47:02.774413 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.774390    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78ddfddc-8h6dt"]
Sep 12 01:47:02.779499 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.779481    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78ddfddc-8h6dt"]
Sep 12 01:47:02.977390 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:02.977362    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" path="/var/lib/kubelet/pods/7bf3f5b9-cebb-46cb-a7a1-8783c44aac95/volumes"
Sep 12 01:47:44.763779 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.763749    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-744d85c9b9-69xsl"]
Sep 12 01:47:44.764482 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.764115    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="865c063b-707c-4239-b057-6194b27e3929" containerName="registry"
Sep 12 01:47:44.764482 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.764132    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="865c063b-707c-4239-b057-6194b27e3929" containerName="registry"
Sep 12 01:47:44.764482 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.764159    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" containerName="console"
Sep 12 01:47:44.764482 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.764167    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" containerName="console"
Sep 12 01:47:44.764482 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.764256    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="865c063b-707c-4239-b057-6194b27e3929" containerName="registry"
Sep 12 01:47:44.764482 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.764269    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf3f5b9-cebb-46cb-a7a1-8783c44aac95" containerName="console"
Sep 12 01:47:44.766573 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.766546    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.776597 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.776573    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744d85c9b9-69xsl"]
Sep 12 01:47:44.842205 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.842177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-trusted-ca-bundle\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.842330 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.842225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvbq6\" (UniqueName: \"kubernetes.io/projected/85d263ca-40a4-45a7-888f-06bfa57c9042-kube-api-access-cvbq6\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.842330 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.842292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-service-ca\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.842330 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.842325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-oauth-serving-cert\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.842465 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.842359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-oauth-config\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.842465 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.842375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-console-config\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.842548 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.842456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-serving-cert\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.943675 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.943650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-service-ca\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.943816 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.943689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-oauth-serving-cert\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.943816 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.943719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-oauth-config\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.943816 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.943735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-console-config\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.943816 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.943766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-serving-cert\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.943816 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.943783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-trusted-ca-bundle\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.943816 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.943800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvbq6\" (UniqueName: \"kubernetes.io/projected/85d263ca-40a4-45a7-888f-06bfa57c9042-kube-api-access-cvbq6\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.944464 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.944442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-service-ca\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.944564 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.944536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-oauth-serving-cert\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.944607 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.944562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-console-config\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.944784 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.944758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-trusted-ca-bundle\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.946197 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.946178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-oauth-config\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.946301 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.946286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-serving-cert\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:44.952307 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:44.952287    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvbq6\" (UniqueName: \"kubernetes.io/projected/85d263ca-40a4-45a7-888f-06bfa57c9042-kube-api-access-cvbq6\") pod \"console-744d85c9b9-69xsl\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") " pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:45.077357 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:45.077269    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:45.348663 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:45.348596    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744d85c9b9-69xsl"]
Sep 12 01:47:45.353075 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:47:45.353046    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d263ca_40a4_45a7_888f_06bfa57c9042.slice/crio-f1e9bf9dd753a7cf27ca0428d336fbc26d57bb3d0d5136877503eb8c6b309361 WatchSource:0}: Error finding container f1e9bf9dd753a7cf27ca0428d336fbc26d57bb3d0d5136877503eb8c6b309361: Status 404 returned error can't find the container with id f1e9bf9dd753a7cf27ca0428d336fbc26d57bb3d0d5136877503eb8c6b309361
Sep 12 01:47:45.918630 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:45.918596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744d85c9b9-69xsl" event={"ID":"85d263ca-40a4-45a7-888f-06bfa57c9042","Type":"ContainerStarted","Data":"4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b"}
Sep 12 01:47:45.918989 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:45.918687    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744d85c9b9-69xsl" event={"ID":"85d263ca-40a4-45a7-888f-06bfa57c9042","Type":"ContainerStarted","Data":"f1e9bf9dd753a7cf27ca0428d336fbc26d57bb3d0d5136877503eb8c6b309361"}
Sep 12 01:47:45.937952 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:45.937908    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-744d85c9b9-69xsl" podStartSLOduration=1.937894432 podStartE2EDuration="1.937894432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:47:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:47:45.937329255 +0000 UTC m=+171.464230336" watchObservedRunningTime="2026-09-12 01:47:45.937894432 +0000 UTC m=+171.464795511"
Sep 12 01:47:55.078398 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:55.078365    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:55.078761 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:55.078417    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:55.083304 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:55.083283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:55.960442 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:55.960409    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:47:56.002195 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:47:56.002165    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-785c467696-jw6cr"]
Sep 12 01:48:21.022528 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.022475    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-785c467696-jw6cr" podUID="c3554fb6-98fa-4280-b755-851379fe9c79" containerName="console" containerID="cri-o://6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033" gracePeriod=15
Sep 12 01:48:21.258374 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.258345    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:48:21.416387 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416303    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-service-ca\") pod \"c3554fb6-98fa-4280-b755-851379fe9c79\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") "
Sep 12 01:48:21.416387 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416340    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrqb2\" (UniqueName: \"kubernetes.io/projected/c3554fb6-98fa-4280-b755-851379fe9c79-kube-api-access-vrqb2\") pod \"c3554fb6-98fa-4280-b755-851379fe9c79\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") "
Sep 12 01:48:21.416387 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416369    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-serving-cert\") pod \"c3554fb6-98fa-4280-b755-851379fe9c79\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") "
Sep 12 01:48:21.416637 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416405    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-oauth-config\") pod \"c3554fb6-98fa-4280-b755-851379fe9c79\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") "
Sep 12 01:48:21.416637 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416447    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-oauth-serving-cert\") pod \"c3554fb6-98fa-4280-b755-851379fe9c79\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") "
Sep 12 01:48:21.416637 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416472    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-console-config\") pod \"c3554fb6-98fa-4280-b755-851379fe9c79\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") "
Sep 12 01:48:21.416637 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416497    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-trusted-ca-bundle\") pod \"c3554fb6-98fa-4280-b755-851379fe9c79\" (UID: \"c3554fb6-98fa-4280-b755-851379fe9c79\") "
Sep 12 01:48:21.416904 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c3554fb6-98fa-4280-b755-851379fe9c79" (UID: "c3554fb6-98fa-4280-b755-851379fe9c79"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:48:21.417001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416969    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-console-config" (OuterVolumeSpecName: "console-config") pod "c3554fb6-98fa-4280-b755-851379fe9c79" (UID: "c3554fb6-98fa-4280-b755-851379fe9c79"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:48:21.417075 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.417002    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-service-ca" (OuterVolumeSpecName: "service-ca") pod "c3554fb6-98fa-4280-b755-851379fe9c79" (UID: "c3554fb6-98fa-4280-b755-851379fe9c79"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:48:21.417075 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.416999    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c3554fb6-98fa-4280-b755-851379fe9c79" (UID: "c3554fb6-98fa-4280-b755-851379fe9c79"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:48:21.418566 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.418536    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3554fb6-98fa-4280-b755-851379fe9c79-kube-api-access-vrqb2" (OuterVolumeSpecName: "kube-api-access-vrqb2") pod "c3554fb6-98fa-4280-b755-851379fe9c79" (UID: "c3554fb6-98fa-4280-b755-851379fe9c79"). InnerVolumeSpecName "kube-api-access-vrqb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:48:21.418566 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.418538    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c3554fb6-98fa-4280-b755-851379fe9c79" (UID: "c3554fb6-98fa-4280-b755-851379fe9c79"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:48:21.418701 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.418575    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c3554fb6-98fa-4280-b755-851379fe9c79" (UID: "c3554fb6-98fa-4280-b755-851379fe9c79"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:48:21.517525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.517487    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-console-config\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:48:21.517525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.517521    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-trusted-ca-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:48:21.517525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.517531    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:48:21.517739 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.517559    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrqb2\" (UniqueName: \"kubernetes.io/projected/c3554fb6-98fa-4280-b755-851379fe9c79-kube-api-access-vrqb2\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:48:21.517739 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.517568    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-serving-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:48:21.517739 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.517576    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3554fb6-98fa-4280-b755-851379fe9c79-console-oauth-config\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:48:21.517739 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:21.517584    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3554fb6-98fa-4280-b755-851379fe9c79-oauth-serving-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:48:22.043901 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.043869    2580 generic.go:356] "Generic (PLEG): container finished" podID="c3554fb6-98fa-4280-b755-851379fe9c79" containerID="6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033" exitCode=0
Sep 12 01:48:22.044305 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.043912    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-785c467696-jw6cr"
Sep 12 01:48:22.044305 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.043920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-785c467696-jw6cr" event={"ID":"c3554fb6-98fa-4280-b755-851379fe9c79","Type":"ContainerDied","Data":"6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033"}
Sep 12 01:48:22.044305 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.043958    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-785c467696-jw6cr" event={"ID":"c3554fb6-98fa-4280-b755-851379fe9c79","Type":"ContainerDied","Data":"181b07496fe56b31e211a655f784bfc1504407d35779f3cc19cfe20c5872029d"}
Sep 12 01:48:22.044305 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.043976    2580 scope.go:117] "RemoveContainer" containerID="6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033"
Sep 12 01:48:22.052456 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.052441    2580 scope.go:117] "RemoveContainer" containerID="6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033"
Sep 12 01:48:22.052718 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:48:22.052700    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033\": container with ID starting with 6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033 not found: ID does not exist" containerID="6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033"
Sep 12 01:48:22.052761 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.052724    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033"} err="failed to get container status \"6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033\": rpc error: code = NotFound desc = could not find container \"6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033\": container with ID starting with 6952fd2e440e34840fcbad2d468e398dbf2653a62d8eeeb085fec344ae3b6033 not found: ID does not exist"
Sep 12 01:48:22.066794 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.066773    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-785c467696-jw6cr"]
Sep 12 01:48:22.071124 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.071106    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-785c467696-jw6cr"]
Sep 12 01:48:22.977423 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:22.977390    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3554fb6-98fa-4280-b755-851379fe9c79" path="/var/lib/kubelet/pods/c3554fb6-98fa-4280-b755-851379fe9c79/volumes"
Sep 12 01:48:52.519633 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.519603    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78f8d6fb4-nx6m6"]
Sep 12 01:48:52.520173 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.520016    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3554fb6-98fa-4280-b755-851379fe9c79" containerName="console"
Sep 12 01:48:52.520173 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.520037    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3554fb6-98fa-4280-b755-851379fe9c79" containerName="console"
Sep 12 01:48:52.520173 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.520126    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3554fb6-98fa-4280-b755-851379fe9c79" containerName="console"
Sep 12 01:48:52.522503 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.522485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.532042 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.532014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-oauth-serving-cert\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.532153 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.532050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-oauth-config\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.532153 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.532115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-serving-cert\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.532389 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.532367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-service-ca\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.532554 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.532517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmcbc\" (UniqueName: \"kubernetes.io/projected/e25e8f41-1278-4e4a-ad36-2f47867ca23e-kube-api-access-bmcbc\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.532659 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.532568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-config\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.532659 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.532592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-trusted-ca-bundle\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.533565 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.533543    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78f8d6fb4-nx6m6"]
Sep 12 01:48:52.633361 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.633332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-serving-cert\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.633361 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.633361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-service-ca\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.633571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.633392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmcbc\" (UniqueName: \"kubernetes.io/projected/e25e8f41-1278-4e4a-ad36-2f47867ca23e-kube-api-access-bmcbc\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.633571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.633419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-config\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.633571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.633443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-trusted-ca-bundle\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.633571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.633493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-oauth-serving-cert\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.633571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.633519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-oauth-config\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.634252 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.634199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-config\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.634386 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.634255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-service-ca\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.634386 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.634332    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-trusted-ca-bundle\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.634386 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.634343    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-oauth-serving-cert\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.635883 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.635856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-serving-cert\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.635973 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.635894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-oauth-config\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.640923 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.640905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmcbc\" (UniqueName: \"kubernetes.io/projected/e25e8f41-1278-4e4a-ad36-2f47867ca23e-kube-api-access-bmcbc\") pod \"console-78f8d6fb4-nx6m6\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") " pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:52.832786 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:52.832703    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:48:53.103736 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:53.103712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78f8d6fb4-nx6m6"]
Sep 12 01:48:53.105644 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:48:53.105619    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25e8f41_1278_4e4a_ad36_2f47867ca23e.slice/crio-df6bddef505b17b074df3735130c1e8a871c38e188f816411a4812161ef34c7b WatchSource:0}: Error finding container df6bddef505b17b074df3735130c1e8a871c38e188f816411a4812161ef34c7b: Status 404 returned error can't find the container with id df6bddef505b17b074df3735130c1e8a871c38e188f816411a4812161ef34c7b
Sep 12 01:48:53.159189 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:53.159161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f8d6fb4-nx6m6" event={"ID":"e25e8f41-1278-4e4a-ad36-2f47867ca23e","Type":"ContainerStarted","Data":"df6bddef505b17b074df3735130c1e8a871c38e188f816411a4812161ef34c7b"}
Sep 12 01:48:54.164657 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:54.164615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f8d6fb4-nx6m6" event={"ID":"e25e8f41-1278-4e4a-ad36-2f47867ca23e","Type":"ContainerStarted","Data":"084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037"}
Sep 12 01:48:54.181164 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:48:54.181116    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78f8d6fb4-nx6m6" podStartSLOduration=2.181102864 podStartE2EDuration="2.181102864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:48:54.180414102 +0000 UTC m=+239.707315206" watchObservedRunningTime="2026-09-12 01:48:54.181102864 +0000 UTC m=+239.708003944"
Sep 12 01:49:02.833197 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:02.833155    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:49:02.833603 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:02.833229    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:49:02.837866 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:02.837845    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:49:03.200379 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:03.200304    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:49:03.243531 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:03.243499    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744d85c9b9-69xsl"]
Sep 12 01:49:28.263735 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.263626    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-744d85c9b9-69xsl" podUID="85d263ca-40a4-45a7-888f-06bfa57c9042" containerName="console" containerID="cri-o://4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b" gracePeriod=15
Sep 12 01:49:28.502014 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.501988    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:49:28.611259 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611134    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-serving-cert\") pod \"85d263ca-40a4-45a7-888f-06bfa57c9042\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") "
Sep 12 01:49:28.611259 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611194    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-oauth-serving-cert\") pod \"85d263ca-40a4-45a7-888f-06bfa57c9042\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") "
Sep 12 01:49:28.611259 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611251    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-console-config\") pod \"85d263ca-40a4-45a7-888f-06bfa57c9042\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") "
Sep 12 01:49:28.611557 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611292    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvbq6\" (UniqueName: \"kubernetes.io/projected/85d263ca-40a4-45a7-888f-06bfa57c9042-kube-api-access-cvbq6\") pod \"85d263ca-40a4-45a7-888f-06bfa57c9042\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") "
Sep 12 01:49:28.611557 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611312    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-trusted-ca-bundle\") pod \"85d263ca-40a4-45a7-888f-06bfa57c9042\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") "
Sep 12 01:49:28.611557 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611338    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-service-ca\") pod \"85d263ca-40a4-45a7-888f-06bfa57c9042\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") "
Sep 12 01:49:28.611557 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611374    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-oauth-config\") pod \"85d263ca-40a4-45a7-888f-06bfa57c9042\" (UID: \"85d263ca-40a4-45a7-888f-06bfa57c9042\") "
Sep 12 01:49:28.611790 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611759    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-console-config" (OuterVolumeSpecName: "console-config") pod "85d263ca-40a4-45a7-888f-06bfa57c9042" (UID: "85d263ca-40a4-45a7-888f-06bfa57c9042"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:49:28.611790 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611780    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "85d263ca-40a4-45a7-888f-06bfa57c9042" (UID: "85d263ca-40a4-45a7-888f-06bfa57c9042"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:49:28.611913 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611774    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "85d263ca-40a4-45a7-888f-06bfa57c9042" (UID: "85d263ca-40a4-45a7-888f-06bfa57c9042"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:49:28.611913 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.611789    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-service-ca" (OuterVolumeSpecName: "service-ca") pod "85d263ca-40a4-45a7-888f-06bfa57c9042" (UID: "85d263ca-40a4-45a7-888f-06bfa57c9042"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:49:28.613455 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.613419    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d263ca-40a4-45a7-888f-06bfa57c9042-kube-api-access-cvbq6" (OuterVolumeSpecName: "kube-api-access-cvbq6") pod "85d263ca-40a4-45a7-888f-06bfa57c9042" (UID: "85d263ca-40a4-45a7-888f-06bfa57c9042"). InnerVolumeSpecName "kube-api-access-cvbq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:49:28.613584 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.613484    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "85d263ca-40a4-45a7-888f-06bfa57c9042" (UID: "85d263ca-40a4-45a7-888f-06bfa57c9042"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:49:28.613627 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.613600    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "85d263ca-40a4-45a7-888f-06bfa57c9042" (UID: "85d263ca-40a4-45a7-888f-06bfa57c9042"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:49:28.712221 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.712182    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-console-config\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:28.712389 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.712236    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvbq6\" (UniqueName: \"kubernetes.io/projected/85d263ca-40a4-45a7-888f-06bfa57c9042-kube-api-access-cvbq6\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:28.712389 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.712247    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-trusted-ca-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:28.712389 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.712257    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:28.712389 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.712265    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-oauth-config\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:28.712389 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.712273    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85d263ca-40a4-45a7-888f-06bfa57c9042-console-serving-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:28.712389 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:28.712281    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85d263ca-40a4-45a7-888f-06bfa57c9042-oauth-serving-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:29.294180 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:29.294148    2580 generic.go:356] "Generic (PLEG): container finished" podID="85d263ca-40a4-45a7-888f-06bfa57c9042" containerID="4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b" exitCode=0
Sep 12 01:49:29.294566 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:29.294197    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744d85c9b9-69xsl"
Sep 12 01:49:29.294566 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:29.294238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744d85c9b9-69xsl" event={"ID":"85d263ca-40a4-45a7-888f-06bfa57c9042","Type":"ContainerDied","Data":"4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b"}
Sep 12 01:49:29.294566 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:29.294300    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744d85c9b9-69xsl" event={"ID":"85d263ca-40a4-45a7-888f-06bfa57c9042","Type":"ContainerDied","Data":"f1e9bf9dd753a7cf27ca0428d336fbc26d57bb3d0d5136877503eb8c6b309361"}
Sep 12 01:49:29.294566 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:29.294328    2580 scope.go:117] "RemoveContainer" containerID="4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b"
Sep 12 01:49:29.302449 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:29.302427    2580 scope.go:117] "RemoveContainer" containerID="4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b"
Sep 12 01:49:29.302729 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:49:29.302711    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b\": container with ID starting with 4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b not found: ID does not exist" containerID="4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b"
Sep 12 01:49:29.302793 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:29.302743    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b"} err="failed to get container status \"4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b\": rpc error: code = NotFound desc = could not find container \"4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b\": container with ID starting with 4c524152122caecfb33f67ed17d347856d5c657307c0c1582f3e319776ea6e5b not found: ID does not exist"
Sep 12 01:49:29.309574 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:29.309549    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744d85c9b9-69xsl"]
Sep 12 01:49:29.312964 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:29.312937    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-744d85c9b9-69xsl"]
Sep 12 01:49:30.977942 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:30.977896    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85d263ca-40a4-45a7-888f-06bfa57c9042" path="/var/lib/kubelet/pods/85d263ca-40a4-45a7-888f-06bfa57c9042/volumes"
Sep 12 01:49:34.826105 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.826046    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"]
Sep 12 01:49:34.826473 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.826379    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85d263ca-40a4-45a7-888f-06bfa57c9042" containerName="console"
Sep 12 01:49:34.826473 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.826392    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="85d263ca-40a4-45a7-888f-06bfa57c9042" containerName="console"
Sep 12 01:49:34.826473 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.826463    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="85d263ca-40a4-45a7-888f-06bfa57c9042" containerName="console"
Sep 12 01:49:34.829467 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.829445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:34.832239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.832220    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 01:49:34.832342 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.832328    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 01:49:34.833414 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.833398    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-txj2m\""
Sep 12 01:49:34.837636 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.837604    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"]
Sep 12 01:49:34.963870 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.963828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:34.963870 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.963867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltzwp\" (UniqueName: \"kubernetes.io/projected/c83de6e3-e32c-4d6e-ab10-74c537b8d625-kube-api-access-ltzwp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:34.964073 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:34.963921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:35.065096 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:35.065054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:35.065310 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:35.065130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:35.065310 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:35.065148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltzwp\" (UniqueName: \"kubernetes.io/projected/c83de6e3-e32c-4d6e-ab10-74c537b8d625-kube-api-access-ltzwp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:35.065519 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:35.065498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:35.065601 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:35.065574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:35.074480 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:35.074455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltzwp\" (UniqueName: \"kubernetes.io/projected/c83de6e3-e32c-4d6e-ab10-74c537b8d625-kube-api-access-ltzwp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:35.139609 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:35.139523    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:35.410877 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:35.410854    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"]
Sep 12 01:49:35.413409 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:49:35.413383    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83de6e3_e32c_4d6e_ab10_74c537b8d625.slice/crio-734edf56424c4f1a3dde4b1ed6bbffbaa926ae07f91f49782d917f2ea1b7740a WatchSource:0}: Error finding container 734edf56424c4f1a3dde4b1ed6bbffbaa926ae07f91f49782d917f2ea1b7740a: Status 404 returned error can't find the container with id 734edf56424c4f1a3dde4b1ed6bbffbaa926ae07f91f49782d917f2ea1b7740a
Sep 12 01:49:36.324140 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:36.324103    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn" event={"ID":"c83de6e3-e32c-4d6e-ab10-74c537b8d625","Type":"ContainerStarted","Data":"734edf56424c4f1a3dde4b1ed6bbffbaa926ae07f91f49782d917f2ea1b7740a"}
Sep 12 01:49:42.350474 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:42.350433    2580 generic.go:356] "Generic (PLEG): container finished" podID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerID="4b44aff6ec5eed0023e8ef3526e6eb642a62cd21aa6f6b06300d8dc87220478b" exitCode=0
Sep 12 01:49:42.350857 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:42.350528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn" event={"ID":"c83de6e3-e32c-4d6e-ab10-74c537b8d625","Type":"ContainerDied","Data":"4b44aff6ec5eed0023e8ef3526e6eb642a62cd21aa6f6b06300d8dc87220478b"}
Sep 12 01:49:45.368816 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:45.368783    2580 generic.go:356] "Generic (PLEG): container finished" podID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerID="2580a68d4653b428ef6fae146edefb58322beda40bace4ba0cc86867b3ea370f" exitCode=0
Sep 12 01:49:45.369240 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:45.368864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn" event={"ID":"c83de6e3-e32c-4d6e-ab10-74c537b8d625","Type":"ContainerDied","Data":"2580a68d4653b428ef6fae146edefb58322beda40bace4ba0cc86867b3ea370f"}
Sep 12 01:49:53.403845 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:53.403807    2580 generic.go:356] "Generic (PLEG): container finished" podID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerID="5d7f7bf19c2956db2fb14ac5f5e2fe3e9c938c3c754a6d047aeee7f73e32af50" exitCode=0
Sep 12 01:49:53.404191 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:53.403873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn" event={"ID":"c83de6e3-e32c-4d6e-ab10-74c537b8d625","Type":"ContainerDied","Data":"5d7f7bf19c2956db2fb14ac5f5e2fe3e9c938c3c754a6d047aeee7f73e32af50"}
Sep 12 01:49:54.533298 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.533272    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:54.625653 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.625616    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-util\") pod \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") "
Sep 12 01:49:54.625835 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.625696    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-bundle\") pod \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") "
Sep 12 01:49:54.625835 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.625736    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltzwp\" (UniqueName: \"kubernetes.io/projected/c83de6e3-e32c-4d6e-ab10-74c537b8d625-kube-api-access-ltzwp\") pod \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\" (UID: \"c83de6e3-e32c-4d6e-ab10-74c537b8d625\") "
Sep 12 01:49:54.626405 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.626373    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-bundle" (OuterVolumeSpecName: "bundle") pod "c83de6e3-e32c-4d6e-ab10-74c537b8d625" (UID: "c83de6e3-e32c-4d6e-ab10-74c537b8d625"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:49:54.628071 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.628039    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c83de6e3-e32c-4d6e-ab10-74c537b8d625-kube-api-access-ltzwp" (OuterVolumeSpecName: "kube-api-access-ltzwp") pod "c83de6e3-e32c-4d6e-ab10-74c537b8d625" (UID: "c83de6e3-e32c-4d6e-ab10-74c537b8d625"). InnerVolumeSpecName "kube-api-access-ltzwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:49:54.629809 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.629783    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-util" (OuterVolumeSpecName: "util") pod "c83de6e3-e32c-4d6e-ab10-74c537b8d625" (UID: "c83de6e3-e32c-4d6e-ab10-74c537b8d625"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:49:54.727183 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.727105    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-util\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:54.727183 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.727135    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c83de6e3-e32c-4d6e-ab10-74c537b8d625-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:54.727183 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.727147    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltzwp\" (UniqueName: \"kubernetes.io/projected/c83de6e3-e32c-4d6e-ab10-74c537b8d625-kube-api-access-ltzwp\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:49:54.888302 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.888269    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 01:49:54.890492 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.890468    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 01:49:54.899339 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:54.899314    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 12 01:49:55.465817 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:55.412598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn" event={"ID":"c83de6e3-e32c-4d6e-ab10-74c537b8d625","Type":"ContainerDied","Data":"734edf56424c4f1a3dde4b1ed6bbffbaa926ae07f91f49782d917f2ea1b7740a"}
Sep 12 01:49:55.465817 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:55.412634    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j29rn"
Sep 12 01:49:55.465817 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:49:55.412645    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="734edf56424c4f1a3dde4b1ed6bbffbaa926ae07f91f49782d917f2ea1b7740a"
Sep 12 01:50:02.756964 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.756928    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"]
Sep 12 01:50:02.757447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.757200    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerName="util"
Sep 12 01:50:02.757447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.757224    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerName="util"
Sep 12 01:50:02.757447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.757236    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerName="extract"
Sep 12 01:50:02.757447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.757244    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerName="extract"
Sep 12 01:50:02.757447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.757255    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerName="pull"
Sep 12 01:50:02.757447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.757261    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerName="pull"
Sep 12 01:50:02.757447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.757310    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c83de6e3-e32c-4d6e-ab10-74c537b8d625" containerName="extract"
Sep 12 01:50:02.759203 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.759187    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"
Sep 12 01:50:02.761850 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.761826    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-prv96\""
Sep 12 01:50:02.761999 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.761884    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 12 01:50:02.761999 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.761972    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 12 01:50:02.771908 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.771881    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"]
Sep 12 01:50:02.776750 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.776725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5jwq\" (UniqueName: \"kubernetes.io/projected/983f1168-0f3f-4604-a947-366305620968-kube-api-access-q5jwq\") pod \"cert-manager-operator-controller-manager-55c444bf86-2bc85\" (UID: \"983f1168-0f3f-4604-a947-366305620968\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"
Sep 12 01:50:02.776862 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.776797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983f1168-0f3f-4604-a947-366305620968-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2bc85\" (UID: \"983f1168-0f3f-4604-a947-366305620968\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"
Sep 12 01:50:02.877505 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.877453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983f1168-0f3f-4604-a947-366305620968-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2bc85\" (UID: \"983f1168-0f3f-4604-a947-366305620968\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"
Sep 12 01:50:02.877679 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.877521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5jwq\" (UniqueName: \"kubernetes.io/projected/983f1168-0f3f-4604-a947-366305620968-kube-api-access-q5jwq\") pod \"cert-manager-operator-controller-manager-55c444bf86-2bc85\" (UID: \"983f1168-0f3f-4604-a947-366305620968\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"
Sep 12 01:50:02.877881 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.877858    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/983f1168-0f3f-4604-a947-366305620968-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2bc85\" (UID: \"983f1168-0f3f-4604-a947-366305620968\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"
Sep 12 01:50:02.886488 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:02.886455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5jwq\" (UniqueName: \"kubernetes.io/projected/983f1168-0f3f-4604-a947-366305620968-kube-api-access-q5jwq\") pod \"cert-manager-operator-controller-manager-55c444bf86-2bc85\" (UID: \"983f1168-0f3f-4604-a947-366305620968\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"
Sep 12 01:50:03.068243 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:03.068135    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"
Sep 12 01:50:03.345244 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:03.345147    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85"]
Sep 12 01:50:03.348757 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:50:03.348727    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983f1168_0f3f_4604_a947_366305620968.slice/crio-a95258a44fc7261af727da52f7e303537c71dbc8a0e9e70113cd3ca7dad1ea64 WatchSource:0}: Error finding container a95258a44fc7261af727da52f7e303537c71dbc8a0e9e70113cd3ca7dad1ea64: Status 404 returned error can't find the container with id a95258a44fc7261af727da52f7e303537c71dbc8a0e9e70113cd3ca7dad1ea64
Sep 12 01:50:03.352138 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:03.352116    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 01:50:03.444718 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:03.444682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85" event={"ID":"983f1168-0f3f-4604-a947-366305620968","Type":"ContainerStarted","Data":"a95258a44fc7261af727da52f7e303537c71dbc8a0e9e70113cd3ca7dad1ea64"}
Sep 12 01:50:06.459127 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.459088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85" event={"ID":"983f1168-0f3f-4604-a947-366305620968","Type":"ContainerStarted","Data":"0aaa599658dc65319b1bf9cd32e4fe859f677328e440da74cb2cf3cca1dcd0ef"}
Sep 12 01:50:06.463619 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.463594    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w527g"]
Sep 12 01:50:06.465722 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.465706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:06.468160 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.468139    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 12 01:50:06.468270 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.468180    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-lh9sd\""
Sep 12 01:50:06.468270 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.468190    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 12 01:50:06.475733 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.475708    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w527g"]
Sep 12 01:50:06.484451 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.484406    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2bc85" podStartSLOduration=2.351404634 podStartE2EDuration="4.484390381s" totalImagesPullingTime="2.132985747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:50:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:50:06.482490793 +0000 UTC m=+312.009391874" watchObservedRunningTime="2026-09-12 01:50:06.484390381 +0000 UTC m=+312.011291459"
Sep 12 01:50:06.506917 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.506886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/083f5bbd-0ef8-4941-bbf7-097793539dbc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w527g\" (UID: \"083f5bbd-0ef8-4941-bbf7-097793539dbc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:06.507063 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.506951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6w4f\" (UniqueName: \"kubernetes.io/projected/083f5bbd-0ef8-4941-bbf7-097793539dbc-kube-api-access-t6w4f\") pod \"cert-manager-webhook-6486ffd997-w527g\" (UID: \"083f5bbd-0ef8-4941-bbf7-097793539dbc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:06.607688 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.607651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6w4f\" (UniqueName: \"kubernetes.io/projected/083f5bbd-0ef8-4941-bbf7-097793539dbc-kube-api-access-t6w4f\") pod \"cert-manager-webhook-6486ffd997-w527g\" (UID: \"083f5bbd-0ef8-4941-bbf7-097793539dbc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:06.607851 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.607702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/083f5bbd-0ef8-4941-bbf7-097793539dbc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w527g\" (UID: \"083f5bbd-0ef8-4941-bbf7-097793539dbc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:06.615563 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.615533    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/083f5bbd-0ef8-4941-bbf7-097793539dbc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w527g\" (UID: \"083f5bbd-0ef8-4941-bbf7-097793539dbc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:06.615785 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.615764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6w4f\" (UniqueName: \"kubernetes.io/projected/083f5bbd-0ef8-4941-bbf7-097793539dbc-kube-api-access-t6w4f\") pod \"cert-manager-webhook-6486ffd997-w527g\" (UID: \"083f5bbd-0ef8-4941-bbf7-097793539dbc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:06.792403 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:06.792366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:07.081404 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.081326    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w527g"]
Sep 12 01:50:07.085320 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:50:07.085283    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod083f5bbd_0ef8_4941_bbf7_097793539dbc.slice/crio-1a243e60eacaaee0e9b319b0942281e5810cfd93f2c1889fd0969166d341fba5 WatchSource:0}: Error finding container 1a243e60eacaaee0e9b319b0942281e5810cfd93f2c1889fd0969166d341fba5: Status 404 returned error can't find the container with id 1a243e60eacaaee0e9b319b0942281e5810cfd93f2c1889fd0969166d341fba5
Sep 12 01:50:07.310085 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.310049    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"]
Sep 12 01:50:07.313009 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.312993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.315632 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.315607    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 01:50:07.315846 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.315830    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-txj2m\""
Sep 12 01:50:07.315935 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.315915    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 01:50:07.325651 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.325627    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"]
Sep 12 01:50:07.414287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.414182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.414287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.414243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.414287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.414274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwszx\" (UniqueName: \"kubernetes.io/projected/388f1759-d46a-400c-aa68-7634c27eb214-kube-api-access-hwszx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.464767 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.464726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w527g" event={"ID":"083f5bbd-0ef8-4941-bbf7-097793539dbc","Type":"ContainerStarted","Data":"1a243e60eacaaee0e9b319b0942281e5810cfd93f2c1889fd0969166d341fba5"}
Sep 12 01:50:07.515153 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.515112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwszx\" (UniqueName: \"kubernetes.io/projected/388f1759-d46a-400c-aa68-7634c27eb214-kube-api-access-hwszx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.515324 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.515234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.515324 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.515284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.515647 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.515627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.515714 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.515669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.526576 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.526542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwszx\" (UniqueName: \"kubernetes.io/projected/388f1759-d46a-400c-aa68-7634c27eb214-kube-api-access-hwszx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.622587 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.622561    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:07.892639 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:07.892614    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"]
Sep 12 01:50:07.894821 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:50:07.894790    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod388f1759_d46a_400c_aa68_7634c27eb214.slice/crio-ef99fa7a5c71b9da4344bc20b3f381c75e47d74e84eb8aeceefe54a284e1e979 WatchSource:0}: Error finding container ef99fa7a5c71b9da4344bc20b3f381c75e47d74e84eb8aeceefe54a284e1e979: Status 404 returned error can't find the container with id ef99fa7a5c71b9da4344bc20b3f381c75e47d74e84eb8aeceefe54a284e1e979
Sep 12 01:50:08.470775 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:08.470738    2580 generic.go:356] "Generic (PLEG): container finished" podID="388f1759-d46a-400c-aa68-7634c27eb214" containerID="68a575c77c8a963a40c56bdad5c47cbaaa8598347167489e8a2789eb3a0f1641" exitCode=0
Sep 12 01:50:08.471248 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:08.470834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8" event={"ID":"388f1759-d46a-400c-aa68-7634c27eb214","Type":"ContainerDied","Data":"68a575c77c8a963a40c56bdad5c47cbaaa8598347167489e8a2789eb3a0f1641"}
Sep 12 01:50:08.471248 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:08.470903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8" event={"ID":"388f1759-d46a-400c-aa68-7634c27eb214","Type":"ContainerStarted","Data":"ef99fa7a5c71b9da4344bc20b3f381c75e47d74e84eb8aeceefe54a284e1e979"}
Sep 12 01:50:09.047185 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.047146    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"]
Sep 12 01:50:09.049432 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.049411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"
Sep 12 01:50:09.051790 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.051763    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-v6bpm\""
Sep 12 01:50:09.059153 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.059118    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"]
Sep 12 01:50:09.128564 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.128527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01f9668c-4597-422a-8ecf-6dd5a747de41-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vkkcs\" (UID: \"01f9668c-4597-422a-8ecf-6dd5a747de41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"
Sep 12 01:50:09.128749 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.128655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt5zr\" (UniqueName: \"kubernetes.io/projected/01f9668c-4597-422a-8ecf-6dd5a747de41-kube-api-access-qt5zr\") pod \"cert-manager-cainjector-559cdd8c87-vkkcs\" (UID: \"01f9668c-4597-422a-8ecf-6dd5a747de41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"
Sep 12 01:50:09.229603 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.229550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01f9668c-4597-422a-8ecf-6dd5a747de41-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vkkcs\" (UID: \"01f9668c-4597-422a-8ecf-6dd5a747de41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"
Sep 12 01:50:09.229792 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.229664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt5zr\" (UniqueName: \"kubernetes.io/projected/01f9668c-4597-422a-8ecf-6dd5a747de41-kube-api-access-qt5zr\") pod \"cert-manager-cainjector-559cdd8c87-vkkcs\" (UID: \"01f9668c-4597-422a-8ecf-6dd5a747de41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"
Sep 12 01:50:09.238368 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.238337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01f9668c-4597-422a-8ecf-6dd5a747de41-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vkkcs\" (UID: \"01f9668c-4597-422a-8ecf-6dd5a747de41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"
Sep 12 01:50:09.238548 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.238371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt5zr\" (UniqueName: \"kubernetes.io/projected/01f9668c-4597-422a-8ecf-6dd5a747de41-kube-api-access-qt5zr\") pod \"cert-manager-cainjector-559cdd8c87-vkkcs\" (UID: \"01f9668c-4597-422a-8ecf-6dd5a747de41\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"
Sep 12 01:50:09.360799 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:09.360713    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"
Sep 12 01:50:10.885778 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:10.885736    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs"]
Sep 12 01:50:10.888394 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:50:10.888366    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01f9668c_4597_422a_8ecf_6dd5a747de41.slice/crio-584dc04964e20cb94d23827ce77e53e2101476dcbe807e3d9ce8047e73f2a1cb WatchSource:0}: Error finding container 584dc04964e20cb94d23827ce77e53e2101476dcbe807e3d9ce8047e73f2a1cb: Status 404 returned error can't find the container with id 584dc04964e20cb94d23827ce77e53e2101476dcbe807e3d9ce8047e73f2a1cb
Sep 12 01:50:11.487226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:11.487171    2580 generic.go:356] "Generic (PLEG): container finished" podID="388f1759-d46a-400c-aa68-7634c27eb214" containerID="75c20e708dee3f2b16d3dba4c2148a81cc4d879e959ae5920e99ef4c1bf625b6" exitCode=0
Sep 12 01:50:11.487423 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:11.487248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8" event={"ID":"388f1759-d46a-400c-aa68-7634c27eb214","Type":"ContainerDied","Data":"75c20e708dee3f2b16d3dba4c2148a81cc4d879e959ae5920e99ef4c1bf625b6"}
Sep 12 01:50:11.489101 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:11.489064    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w527g" event={"ID":"083f5bbd-0ef8-4941-bbf7-097793539dbc","Type":"ContainerStarted","Data":"90a6d053a23423faf9299a0cb56f96f567056950d50c82142f1b0f9586219117"}
Sep 12 01:50:11.489248 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:11.489111    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:11.490778 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:11.490757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs" event={"ID":"01f9668c-4597-422a-8ecf-6dd5a747de41","Type":"ContainerStarted","Data":"2c3c3b7e44e67cf8607abcf78e87132c8564d39248517d8512595faced650b09"}
Sep 12 01:50:11.490873 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:11.490789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs" event={"ID":"01f9668c-4597-422a-8ecf-6dd5a747de41","Type":"ContainerStarted","Data":"584dc04964e20cb94d23827ce77e53e2101476dcbe807e3d9ce8047e73f2a1cb"}
Sep 12 01:50:11.518168 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:11.518124    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkkcs" podStartSLOduration=2.518109335 podStartE2EDuration="2.518109335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:50:11.517328497 +0000 UTC m=+317.044229578" watchObservedRunningTime="2026-09-12 01:50:11.518109335 +0000 UTC m=+317.045010414"
Sep 12 01:50:11.537778 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:11.537736    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-w527g" podStartSLOduration=2.001862834 podStartE2EDuration="5.537720071s" totalImagesPullingTime="3.535857237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:50:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:50:11.536175249 +0000 UTC m=+317.063076330" watchObservedRunningTime="2026-09-12 01:50:11.537720071 +0000 UTC m=+317.064621151"
Sep 12 01:50:12.498030 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:12.497990    2580 generic.go:356] "Generic (PLEG): container finished" podID="388f1759-d46a-400c-aa68-7634c27eb214" containerID="ce03cf38c3c3c2a4b53c3b41999f615722aed24cd137c4daed59164dc7b31040" exitCode=0
Sep 12 01:50:12.498549 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:12.498073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8" event={"ID":"388f1759-d46a-400c-aa68-7634c27eb214","Type":"ContainerDied","Data":"ce03cf38c3c3c2a4b53c3b41999f615722aed24cd137c4daed59164dc7b31040"}
Sep 12 01:50:13.631160 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.631137    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:13.669418 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.669387    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-util\") pod \"388f1759-d46a-400c-aa68-7634c27eb214\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") "
Sep 12 01:50:13.669571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.669438    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-bundle\") pod \"388f1759-d46a-400c-aa68-7634c27eb214\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") "
Sep 12 01:50:13.669571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.669495    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwszx\" (UniqueName: \"kubernetes.io/projected/388f1759-d46a-400c-aa68-7634c27eb214-kube-api-access-hwszx\") pod \"388f1759-d46a-400c-aa68-7634c27eb214\" (UID: \"388f1759-d46a-400c-aa68-7634c27eb214\") "
Sep 12 01:50:13.669883 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.669859    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-bundle" (OuterVolumeSpecName: "bundle") pod "388f1759-d46a-400c-aa68-7634c27eb214" (UID: "388f1759-d46a-400c-aa68-7634c27eb214"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:50:13.671615 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.671581    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/388f1759-d46a-400c-aa68-7634c27eb214-kube-api-access-hwszx" (OuterVolumeSpecName: "kube-api-access-hwszx") pod "388f1759-d46a-400c-aa68-7634c27eb214" (UID: "388f1759-d46a-400c-aa68-7634c27eb214"). InnerVolumeSpecName "kube-api-access-hwszx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:50:13.676960 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.676916    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-util" (OuterVolumeSpecName: "util") pod "388f1759-d46a-400c-aa68-7634c27eb214" (UID: "388f1759-d46a-400c-aa68-7634c27eb214"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:50:13.770627 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.770524    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwszx\" (UniqueName: \"kubernetes.io/projected/388f1759-d46a-400c-aa68-7634c27eb214-kube-api-access-hwszx\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:50:13.770627 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.770569    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-util\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:50:13.770627 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:13.770583    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/388f1759-d46a-400c-aa68-7634c27eb214-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:50:14.508421 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:14.508392    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8"
Sep 12 01:50:14.508562 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:14.508392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6ghs8" event={"ID":"388f1759-d46a-400c-aa68-7634c27eb214","Type":"ContainerDied","Data":"ef99fa7a5c71b9da4344bc20b3f381c75e47d74e84eb8aeceefe54a284e1e979"}
Sep 12 01:50:14.508562 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:14.508510    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef99fa7a5c71b9da4344bc20b3f381c75e47d74e84eb8aeceefe54a284e1e979"
Sep 12 01:50:17.501134 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:17.501100    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w527g"
Sep 12 01:50:32.847819 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.847787    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"]
Sep 12 01:50:32.848249 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.848068    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="388f1759-d46a-400c-aa68-7634c27eb214" containerName="extract"
Sep 12 01:50:32.848249 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.848079    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="388f1759-d46a-400c-aa68-7634c27eb214" containerName="extract"
Sep 12 01:50:32.848249 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.848087    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="388f1759-d46a-400c-aa68-7634c27eb214" containerName="pull"
Sep 12 01:50:32.848249 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.848092    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="388f1759-d46a-400c-aa68-7634c27eb214" containerName="pull"
Sep 12 01:50:32.848249 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.848098    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="388f1759-d46a-400c-aa68-7634c27eb214" containerName="util"
Sep 12 01:50:32.848249 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.848104    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="388f1759-d46a-400c-aa68-7634c27eb214" containerName="util"
Sep 12 01:50:32.848249 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.848153    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="388f1759-d46a-400c-aa68-7634c27eb214" containerName="extract"
Sep 12 01:50:32.852677 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.852661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:32.855297 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.855274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 01:50:32.855491 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.855332    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-txj2m\""
Sep 12 01:50:32.856363 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.856342    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 01:50:32.858583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.858561    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"]
Sep 12 01:50:32.930279 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.930255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:32.930408 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.930286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:32.930408 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:32.930304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h57vw\" (UniqueName: \"kubernetes.io/projected/6f67d574-246f-428c-9b86-c39f537f5d50-kube-api-access-h57vw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:33.031591 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.031564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:33.031729 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.031593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:33.031729 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.031613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h57vw\" (UniqueName: \"kubernetes.io/projected/6f67d574-246f-428c-9b86-c39f537f5d50-kube-api-access-h57vw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:33.031997 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.031975    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:33.032059 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.031995    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:33.040525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.040500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h57vw\" (UniqueName: \"kubernetes.io/projected/6f67d574-246f-428c-9b86-c39f537f5d50-kube-api-access-h57vw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:33.163105 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.163049    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:33.432155 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.432081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"]
Sep 12 01:50:33.435574 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:50:33.435550    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f67d574_246f_428c_9b86_c39f537f5d50.slice/crio-c9dd52f3831a6e17921bcd1a6b3b5971d04ab79446189be9a64c2d10d2632517 WatchSource:0}: Error finding container c9dd52f3831a6e17921bcd1a6b3b5971d04ab79446189be9a64c2d10d2632517: Status 404 returned error can't find the container with id c9dd52f3831a6e17921bcd1a6b3b5971d04ab79446189be9a64c2d10d2632517
Sep 12 01:50:33.598646 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.598612    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f67d574-246f-428c-9b86-c39f537f5d50" containerID="02d29d66049ceba66d2d7ce289f3df456ddb034dfeb438a3c8aa1e5e290210fa" exitCode=0
Sep 12 01:50:33.598785 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.598712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5" event={"ID":"6f67d574-246f-428c-9b86-c39f537f5d50","Type":"ContainerDied","Data":"02d29d66049ceba66d2d7ce289f3df456ddb034dfeb438a3c8aa1e5e290210fa"}
Sep 12 01:50:33.598785 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:33.598768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5" event={"ID":"6f67d574-246f-428c-9b86-c39f537f5d50","Type":"ContainerStarted","Data":"c9dd52f3831a6e17921bcd1a6b3b5971d04ab79446189be9a64c2d10d2632517"}
Sep 12 01:50:34.604728 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:34.604691    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f67d574-246f-428c-9b86-c39f537f5d50" containerID="5ecdb9867ba6214e1ba7710c679cee2148f10fe047ae7a7982300cd0ff857be3" exitCode=0
Sep 12 01:50:34.605104 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:34.604775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5" event={"ID":"6f67d574-246f-428c-9b86-c39f537f5d50","Type":"ContainerDied","Data":"5ecdb9867ba6214e1ba7710c679cee2148f10fe047ae7a7982300cd0ff857be3"}
Sep 12 01:50:35.611416 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:35.611380    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f67d574-246f-428c-9b86-c39f537f5d50" containerID="3098c1e6a68aa66ebeaa977807206676f76515a95287cc0d353b011259413f62" exitCode=0
Sep 12 01:50:35.611839 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:35.611463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5" event={"ID":"6f67d574-246f-428c-9b86-c39f537f5d50","Type":"ContainerDied","Data":"3098c1e6a68aa66ebeaa977807206676f76515a95287cc0d353b011259413f62"}
Sep 12 01:50:36.737707 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.737684    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:36.862108 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.862073    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h57vw\" (UniqueName: \"kubernetes.io/projected/6f67d574-246f-428c-9b86-c39f537f5d50-kube-api-access-h57vw\") pod \"6f67d574-246f-428c-9b86-c39f537f5d50\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") "
Sep 12 01:50:36.862270 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.862135    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-bundle\") pod \"6f67d574-246f-428c-9b86-c39f537f5d50\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") "
Sep 12 01:50:36.862270 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.862158    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-util\") pod \"6f67d574-246f-428c-9b86-c39f537f5d50\" (UID: \"6f67d574-246f-428c-9b86-c39f537f5d50\") "
Sep 12 01:50:36.862910 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.862888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-bundle" (OuterVolumeSpecName: "bundle") pod "6f67d574-246f-428c-9b86-c39f537f5d50" (UID: "6f67d574-246f-428c-9b86-c39f537f5d50"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:50:36.864278 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.864256    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f67d574-246f-428c-9b86-c39f537f5d50-kube-api-access-h57vw" (OuterVolumeSpecName: "kube-api-access-h57vw") pod "6f67d574-246f-428c-9b86-c39f537f5d50" (UID: "6f67d574-246f-428c-9b86-c39f537f5d50"). InnerVolumeSpecName "kube-api-access-h57vw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:50:36.867691 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.867628    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-util" (OuterVolumeSpecName: "util") pod "6f67d574-246f-428c-9b86-c39f537f5d50" (UID: "6f67d574-246f-428c-9b86-c39f537f5d50"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:50:36.963113 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.963080    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:50:36.963113 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.963101    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f67d574-246f-428c-9b86-c39f537f5d50-util\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:50:36.963113 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:36.963110    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h57vw\" (UniqueName: \"kubernetes.io/projected/6f67d574-246f-428c-9b86-c39f537f5d50-kube-api-access-h57vw\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:50:37.622344 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:37.622310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5" event={"ID":"6f67d574-246f-428c-9b86-c39f537f5d50","Type":"ContainerDied","Data":"c9dd52f3831a6e17921bcd1a6b3b5971d04ab79446189be9a64c2d10d2632517"}
Sep 12 01:50:37.622508 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:37.622357    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9trw5"
Sep 12 01:50:37.622508 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:37.622355    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9dd52f3831a6e17921bcd1a6b3b5971d04ab79446189be9a64c2d10d2632517"
Sep 12 01:50:42.875907 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.875863    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"]
Sep 12 01:50:42.876442 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.876287    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f67d574-246f-428c-9b86-c39f537f5d50" containerName="util"
Sep 12 01:50:42.876442 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.876306    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f67d574-246f-428c-9b86-c39f537f5d50" containerName="util"
Sep 12 01:50:42.876442 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.876334    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f67d574-246f-428c-9b86-c39f537f5d50" containerName="pull"
Sep 12 01:50:42.876442 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.876342    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f67d574-246f-428c-9b86-c39f537f5d50" containerName="pull"
Sep 12 01:50:42.876442 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.876351    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f67d574-246f-428c-9b86-c39f537f5d50" containerName="extract"
Sep 12 01:50:42.876442 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.876360    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f67d574-246f-428c-9b86-c39f537f5d50" containerName="extract"
Sep 12 01:50:42.876442 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.876432    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f67d574-246f-428c-9b86-c39f537f5d50" containerName="extract"
Sep 12 01:50:42.879488 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.879469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:42.882565 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.882545    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-txj2m\""
Sep 12 01:50:42.882684 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.882583    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 01:50:42.882684 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.882611    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 01:50:42.888136 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:42.888115    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"]
Sep 12 01:50:43.013137 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.013098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.013137 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.013142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.013383 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.013161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqqsq\" (UniqueName: \"kubernetes.io/projected/41c76cba-ab64-4b35-9084-e688d864a7c4-kube-api-access-jqqsq\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.114333 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.114295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.114333 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.114338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.114559 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.114356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqqsq\" (UniqueName: \"kubernetes.io/projected/41c76cba-ab64-4b35-9084-e688d864a7c4-kube-api-access-jqqsq\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.114706 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.114687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.114742 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.114702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.125655 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.125626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqqsq\" (UniqueName: \"kubernetes.io/projected/41c76cba-ab64-4b35-9084-e688d864a7c4-kube-api-access-jqqsq\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.189473 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.189363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:43.463809 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.463780    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"]
Sep 12 01:50:43.466428 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:50:43.466400    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c76cba_ab64_4b35_9084_e688d864a7c4.slice/crio-bfa4effaed41e6dc3ad06f9edfe7ce8ee9415ab63518e40511fec3e3ec0100d0 WatchSource:0}: Error finding container bfa4effaed41e6dc3ad06f9edfe7ce8ee9415ab63518e40511fec3e3ec0100d0: Status 404 returned error can't find the container with id bfa4effaed41e6dc3ad06f9edfe7ce8ee9415ab63518e40511fec3e3ec0100d0
Sep 12 01:50:43.651248 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.651192    2580 generic.go:356] "Generic (PLEG): container finished" podID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerID="b97d2b0d53ea6072955386f60d44b1f2eb2963d2a26a6e7d3169d5efbfafc883" exitCode=0
Sep 12 01:50:43.651419 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.651283    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2" event={"ID":"41c76cba-ab64-4b35-9084-e688d864a7c4","Type":"ContainerDied","Data":"b97d2b0d53ea6072955386f60d44b1f2eb2963d2a26a6e7d3169d5efbfafc883"}
Sep 12 01:50:43.651419 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:43.651335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2" event={"ID":"41c76cba-ab64-4b35-9084-e688d864a7c4","Type":"ContainerStarted","Data":"bfa4effaed41e6dc3ad06f9edfe7ce8ee9415ab63518e40511fec3e3ec0100d0"}
Sep 12 01:50:44.657636 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.657598    2580 generic.go:356] "Generic (PLEG): container finished" podID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerID="b4224837940efae6cfc64be17f2b66f1254400b867f099c9dfd7f70ffd91e849" exitCode=0
Sep 12 01:50:44.658105 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.657657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2" event={"ID":"41c76cba-ab64-4b35-9084-e688d864a7c4","Type":"ContainerDied","Data":"b4224837940efae6cfc64be17f2b66f1254400b867f099c9dfd7f70ffd91e849"}
Sep 12 01:50:44.966143 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.966061    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"]
Sep 12 01:50:44.968441 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.968423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:44.971499 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.971477    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 12 01:50:44.971675 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.971652    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 12 01:50:44.971738 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.971674    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qhcs5\""
Sep 12 01:50:44.971784 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.971752    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 12 01:50:44.971939 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.971922    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 12 01:50:44.990476 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:44.990440    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"]
Sep 12 01:50:45.133685 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.133640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/538d7564-cd35-4560-b351-0ad5beb7a992-manifests\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.133879 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.133711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm9dt\" (UniqueName: \"kubernetes.io/projected/538d7564-cd35-4560-b351-0ad5beb7a992-kube-api-access-fm9dt\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.133879 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.133815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/538d7564-cd35-4560-b351-0ad5beb7a992-tmp\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.133963 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.133889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/538d7564-cd35-4560-b351-0ad5beb7a992-webhook-cert\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.133963 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.133926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/538d7564-cd35-4560-b351-0ad5beb7a992-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.235071 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.234989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/538d7564-cd35-4560-b351-0ad5beb7a992-tmp\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.235071 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.235035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/538d7564-cd35-4560-b351-0ad5beb7a992-webhook-cert\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.235071 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.235057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/538d7564-cd35-4560-b351-0ad5beb7a992-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.235308 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.235078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/538d7564-cd35-4560-b351-0ad5beb7a992-manifests\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.235308 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.235122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9dt\" (UniqueName: \"kubernetes.io/projected/538d7564-cd35-4560-b351-0ad5beb7a992-kube-api-access-fm9dt\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.235585 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.235557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/538d7564-cd35-4560-b351-0ad5beb7a992-manifests\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.237509 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.237476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/538d7564-cd35-4560-b351-0ad5beb7a992-tmp\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.237758 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.237679    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/538d7564-cd35-4560-b351-0ad5beb7a992-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.237758 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.237746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/538d7564-cd35-4560-b351-0ad5beb7a992-webhook-cert\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.247996 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.247968    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9dt\" (UniqueName: \"kubernetes.io/projected/538d7564-cd35-4560-b351-0ad5beb7a992-kube-api-access-fm9dt\") pod \"opendatahub-operator-controller-manager-578c78994-z59bz\" (UID: \"538d7564-cd35-4560-b351-0ad5beb7a992\") " pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.279793 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.279753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:45.558454 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.558416    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"]
Sep 12 01:50:45.561708 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:50:45.561680    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538d7564_cd35_4560_b351_0ad5beb7a992.slice/crio-6d6c87c1626309bd98eb4353037ddf8af9e9c7d63581fe9962c4494c58ee2230 WatchSource:0}: Error finding container 6d6c87c1626309bd98eb4353037ddf8af9e9c7d63581fe9962c4494c58ee2230: Status 404 returned error can't find the container with id 6d6c87c1626309bd98eb4353037ddf8af9e9c7d63581fe9962c4494c58ee2230
Sep 12 01:50:45.665399 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.665366    2580 generic.go:356] "Generic (PLEG): container finished" podID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerID="14dfbf476f8ce0628b3809760a7e94e81d9bbd41ada19608f900006df26578d3" exitCode=0
Sep 12 01:50:45.665842 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.665468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2" event={"ID":"41c76cba-ab64-4b35-9084-e688d864a7c4","Type":"ContainerDied","Data":"14dfbf476f8ce0628b3809760a7e94e81d9bbd41ada19608f900006df26578d3"}
Sep 12 01:50:45.666865 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:45.666841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz" event={"ID":"538d7564-cd35-4560-b351-0ad5beb7a992","Type":"ContainerStarted","Data":"6d6c87c1626309bd98eb4353037ddf8af9e9c7d63581fe9962c4494c58ee2230"}
Sep 12 01:50:46.828874 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:46.828681    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:46.949700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:46.949603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqqsq\" (UniqueName: \"kubernetes.io/projected/41c76cba-ab64-4b35-9084-e688d864a7c4-kube-api-access-jqqsq\") pod \"41c76cba-ab64-4b35-9084-e688d864a7c4\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") "
Sep 12 01:50:46.949859 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:46.949720    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-util\") pod \"41c76cba-ab64-4b35-9084-e688d864a7c4\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") "
Sep 12 01:50:46.949859 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:46.949764    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-bundle\") pod \"41c76cba-ab64-4b35-9084-e688d864a7c4\" (UID: \"41c76cba-ab64-4b35-9084-e688d864a7c4\") "
Sep 12 01:50:46.950995 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:46.950796    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-bundle" (OuterVolumeSpecName: "bundle") pod "41c76cba-ab64-4b35-9084-e688d864a7c4" (UID: "41c76cba-ab64-4b35-9084-e688d864a7c4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:50:46.952518 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:46.952467    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41c76cba-ab64-4b35-9084-e688d864a7c4-kube-api-access-jqqsq" (OuterVolumeSpecName: "kube-api-access-jqqsq") pod "41c76cba-ab64-4b35-9084-e688d864a7c4" (UID: "41c76cba-ab64-4b35-9084-e688d864a7c4"). InnerVolumeSpecName "kube-api-access-jqqsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:50:46.957603 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:46.957573    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-util" (OuterVolumeSpecName: "util") pod "41c76cba-ab64-4b35-9084-e688d864a7c4" (UID: "41c76cba-ab64-4b35-9084-e688d864a7c4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:50:47.050496 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:47.050459    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqqsq\" (UniqueName: \"kubernetes.io/projected/41c76cba-ab64-4b35-9084-e688d864a7c4-kube-api-access-jqqsq\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:50:47.050496 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:47.050491    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-util\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:50:47.050496 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:47.050501    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41c76cba-ab64-4b35-9084-e688d864a7c4-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:50:47.679647 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:47.679567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2" event={"ID":"41c76cba-ab64-4b35-9084-e688d864a7c4","Type":"ContainerDied","Data":"bfa4effaed41e6dc3ad06f9edfe7ce8ee9415ab63518e40511fec3e3ec0100d0"}
Sep 12 01:50:47.679647 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:47.679615    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfa4effaed41e6dc3ad06f9edfe7ce8ee9415ab63518e40511fec3e3ec0100d0"
Sep 12 01:50:47.679647 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:47.679624    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pmmg2"
Sep 12 01:50:48.685624 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:48.685585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz" event={"ID":"538d7564-cd35-4560-b351-0ad5beb7a992","Type":"ContainerStarted","Data":"cfe13bf07828971c54aeb641a176d9c16c72d4b37c7884126b6e6d7c4a3dcb73"}
Sep 12 01:50:49.691430 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:49.691386    2580 generic.go:356] "Generic (PLEG): container finished" podID="538d7564-cd35-4560-b351-0ad5beb7a992" containerID="cfe13bf07828971c54aeb641a176d9c16c72d4b37c7884126b6e6d7c4a3dcb73" exitCode=0
Sep 12 01:50:49.691888 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:49.691468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz" event={"ID":"538d7564-cd35-4560-b351-0ad5beb7a992","Type":"ContainerDied","Data":"cfe13bf07828971c54aeb641a176d9c16c72d4b37c7884126b6e6d7c4a3dcb73"}
Sep 12 01:50:50.698973 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:50.698934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz" event={"ID":"538d7564-cd35-4560-b351-0ad5beb7a992","Type":"ContainerStarted","Data":"1d9b645783201cb80654804e0656fdcf19136ec75d2abcccc855768ca559c149"}
Sep 12 01:50:50.699457 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:50.699081    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:50:50.721501 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:50:50.721442    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz" podStartSLOduration=2.364357821 podStartE2EDuration="6.721423552s" totalImagesPullingTime="3.307057797s" totalInitContainerRuntime="1.050007934s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:50:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:50:50.720456379 +0000 UTC m=+356.247357453" watchObservedRunningTime="2026-09-12 01:50:50.721423552 +0000 UTC m=+356.248324665"
Sep 12 01:51:01.705822 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:01.705790    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-578c78994-z59bz"
Sep 12 01:51:04.766368 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.766334    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"]
Sep 12 01:51:04.766826 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.766629    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerName="pull"
Sep 12 01:51:04.766826 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.766642    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerName="pull"
Sep 12 01:51:04.766826 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.766656    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerName="util"
Sep 12 01:51:04.766826 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.766661    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerName="util"
Sep 12 01:51:04.766826 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.766674    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerName="extract"
Sep 12 01:51:04.766826 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.766679    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerName="extract"
Sep 12 01:51:04.766826 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.766727    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="41c76cba-ab64-4b35-9084-e688d864a7c4" containerName="extract"
Sep 12 01:51:04.879837 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.879803    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"]
Sep 12 01:51:04.879997 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.879901    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:04.884408 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.884382    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 12 01:51:04.884585 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.884425    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7rmgc\""
Sep 12 01:51:04.884585 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.884455    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 12 01:51:04.884585 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.884382    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 12 01:51:04.885426 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.885401    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 12 01:51:04.885583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.885565    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 12 01:51:04.902126 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.902101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dd05d2c9-02a0-45d3-80df-880ef6059fc3-manager-config\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:04.902231 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.902136    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dd05d2c9-02a0-45d3-80df-880ef6059fc3-metrics-cert\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:04.902231 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.902155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt5kk\" (UniqueName: \"kubernetes.io/projected/dd05d2c9-02a0-45d3-80df-880ef6059fc3-kube-api-access-xt5kk\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:04.902321 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:04.902253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd05d2c9-02a0-45d3-80df-880ef6059fc3-cert\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.002708 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.002671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dd05d2c9-02a0-45d3-80df-880ef6059fc3-metrics-cert\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.002708 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.002705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt5kk\" (UniqueName: \"kubernetes.io/projected/dd05d2c9-02a0-45d3-80df-880ef6059fc3-kube-api-access-xt5kk\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.002967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.002748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd05d2c9-02a0-45d3-80df-880ef6059fc3-cert\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.002967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.002791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dd05d2c9-02a0-45d3-80df-880ef6059fc3-manager-config\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.003581 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.003550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dd05d2c9-02a0-45d3-80df-880ef6059fc3-manager-config\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.005609 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.005578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd05d2c9-02a0-45d3-80df-880ef6059fc3-cert\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.005731 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.005667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dd05d2c9-02a0-45d3-80df-880ef6059fc3-metrics-cert\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.030638 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.030510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt5kk\" (UniqueName: \"kubernetes.io/projected/dd05d2c9-02a0-45d3-80df-880ef6059fc3-kube-api-access-xt5kk\") pod \"lws-controller-manager-b9f9494c8-njtqr\" (UID: \"dd05d2c9-02a0-45d3-80df-880ef6059fc3\") " pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.189909 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.189869    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:05.477470 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.474606    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"]
Sep 12 01:51:05.478083 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:51:05.478056    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd05d2c9_02a0_45d3_80df_880ef6059fc3.slice/crio-d0e112809f4d9a21efa309ff167ac420f93fd6c69bbe93f3784fdebc2f51fdbb WatchSource:0}: Error finding container d0e112809f4d9a21efa309ff167ac420f93fd6c69bbe93f3784fdebc2f51fdbb: Status 404 returned error can't find the container with id d0e112809f4d9a21efa309ff167ac420f93fd6c69bbe93f3784fdebc2f51fdbb
Sep 12 01:51:05.772849 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:05.772820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr" event={"ID":"dd05d2c9-02a0-45d3-80df-880ef6059fc3","Type":"ContainerStarted","Data":"d0e112809f4d9a21efa309ff167ac420f93fd6c69bbe93f3784fdebc2f51fdbb"}
Sep 12 01:51:07.784943 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:07.784906    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr" event={"ID":"dd05d2c9-02a0-45d3-80df-880ef6059fc3","Type":"ContainerStarted","Data":"206e7fe2a0c92c9639b833cac257fcc5f81303654beb01bd297e755da94bf1e5"}
Sep 12 01:51:07.785365 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:07.785019    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:07.802146 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:07.802095    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr" podStartSLOduration=1.951869968 podStartE2EDuration="3.802075284s" totalImagesPullingTime="1.850205316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:51:07.800352673 +0000 UTC m=+373.327253754" watchObservedRunningTime="2026-09-12 01:51:07.802075284 +0000 UTC m=+373.328976366"
Sep 12 01:51:11.462949 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.462910    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"]
Sep 12 01:51:11.465345 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.465323    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.467932 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.467894    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 12 01:51:11.468056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.467909    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kx296\""
Sep 12 01:51:11.489230 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.479521    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"]
Sep 12 01:51:11.558833 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.558789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bchj7\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-kube-api-access-bchj7\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.558833 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.558840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.559042 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.558870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c421b-3698-49dc-a82e-75e7be8c109f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.559042 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.558911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.559042 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.558938    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.559042 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.558983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.559042 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.559010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.559042 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.559029    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.559266 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.559052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.660499 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.660455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.660700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.660515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.660700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.660543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.660700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.660580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.660700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.660625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchj7\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-kube-api-access-bchj7\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.660700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.660669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.660700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.660700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c421b-3698-49dc-a82e-75e7be8c109f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.661011 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.660728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.661011 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.660858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.661107 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.661017    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.661107 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.661045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.661295 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.661277    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.661372 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.661334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.661522 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.661498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c421b-3698-49dc-a82e-75e7be8c109f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.663007 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.662985    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.663176 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.663157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.700038 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.700004    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bchj7\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-kube-api-access-bchj7\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.700185 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.700011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.747773 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.747736    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"]
Sep 12 01:51:11.750838 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.750817    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.764924 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.764899    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"]
Sep 12 01:51:11.780284 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.780256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:11.863306 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.863255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.863472 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.863322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.863472 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.863378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.863472 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.863408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.863472 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.863437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pdqm\" (UniqueName: \"kubernetes.io/projected/998fe068-bc7f-4aae-8f7c-d87e468c5db8-kube-api-access-2pdqm\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.863612 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.863518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.863612 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.863576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.863612 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.863597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.863701 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.863617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965034 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.964994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965034 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965306 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdqm\" (UniqueName: \"kubernetes.io/projected/998fe068-bc7f-4aae-8f7c-d87e468c5db8-kube-api-access-2pdqm\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965306 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965306 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965306 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965306 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965306 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965306 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965654 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965709 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965767 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965704    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965858 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.965910 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.965877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.967532 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.967502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.967725 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.967709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.976052 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.976027    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/998fe068-bc7f-4aae-8f7c-d87e468c5db8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:11.976146 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:11.976087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pdqm\" (UniqueName: \"kubernetes.io/projected/998fe068-bc7f-4aae-8f7c-d87e468c5db8-kube-api-access-2pdqm\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn\" (UID: \"998fe068-bc7f-4aae-8f7c-d87e468c5db8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:12.057238 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:12.057168    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"]
Sep 12 01:51:12.059502 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:51:12.059477    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d2c421b_3698_49dc_a82e_75e7be8c109f.slice/crio-e73abf51968e87fb00ba6b431a07f00ccd5d5b43bf55d7b0e165da80849090f8 WatchSource:0}: Error finding container e73abf51968e87fb00ba6b431a07f00ccd5d5b43bf55d7b0e165da80849090f8: Status 404 returned error can't find the container with id e73abf51968e87fb00ba6b431a07f00ccd5d5b43bf55d7b0e165da80849090f8
Sep 12 01:51:12.060875 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:12.060857    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:12.337053 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:12.337026    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"]
Sep 12 01:51:12.338883 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:51:12.338854    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998fe068_bc7f_4aae_8f7c_d87e468c5db8.slice/crio-1f8760f26f5ef0c9403c3cc1985c6b9c265b0ff9bd46ccb0fe95f5c609bc1e9b WatchSource:0}: Error finding container 1f8760f26f5ef0c9403c3cc1985c6b9c265b0ff9bd46ccb0fe95f5c609bc1e9b: Status 404 returned error can't find the container with id 1f8760f26f5ef0c9403c3cc1985c6b9c265b0ff9bd46ccb0fe95f5c609bc1e9b
Sep 12 01:51:12.810800 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:12.810761    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn" event={"ID":"998fe068-bc7f-4aae-8f7c-d87e468c5db8","Type":"ContainerStarted","Data":"1f8760f26f5ef0c9403c3cc1985c6b9c265b0ff9bd46ccb0fe95f5c609bc1e9b"}
Sep 12 01:51:12.812386 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:12.812356    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w" event={"ID":"6d2c421b-3698-49dc-a82e-75e7be8c109f","Type":"ContainerStarted","Data":"e73abf51968e87fb00ba6b431a07f00ccd5d5b43bf55d7b0e165da80849090f8"}
Sep 12 01:51:14.885659 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:14.885605    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:51:14.886040 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:14.885714    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:51:14.886040 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:14.885794    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:51:14.891932 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:14.891896    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:51:14.892006 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:14.891971    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:51:14.892048 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:14.892007    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:51:15.514030 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.513996    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-688bf79bb6-8v5w4"]
Sep 12 01:51:15.516591 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.516557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.528068 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.528039    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688bf79bb6-8v5w4"]
Sep 12 01:51:15.602687 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.602654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-service-ca\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.602878 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.602702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-console-config\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.602878 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.602733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fec9c0b-98aa-414b-abe5-9483d44f32a4-console-oauth-config\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.602878 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.602798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-oauth-serving-cert\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.602878 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.602863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgtcz\" (UniqueName: \"kubernetes.io/projected/0fec9c0b-98aa-414b-abe5-9483d44f32a4-kube-api-access-zgtcz\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.603022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.602898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fec9c0b-98aa-414b-abe5-9483d44f32a4-console-serving-cert\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.603022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.602927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-trusted-ca-bundle\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.703845 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.703806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fec9c0b-98aa-414b-abe5-9483d44f32a4-console-serving-cert\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.704022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.703854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-trusted-ca-bundle\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.704022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.703895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-service-ca\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.704022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.703940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-console-config\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.704022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.703989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fec9c0b-98aa-414b-abe5-9483d44f32a4-console-oauth-config\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.704241 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.704040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-oauth-serving-cert\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.704241 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.704094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgtcz\" (UniqueName: \"kubernetes.io/projected/0fec9c0b-98aa-414b-abe5-9483d44f32a4-kube-api-access-zgtcz\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.704783 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.704761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-console-config\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.704886 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.704832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-trusted-ca-bundle\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.704964 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.704939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-service-ca\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.705003 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.704959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fec9c0b-98aa-414b-abe5-9483d44f32a4-oauth-serving-cert\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.706516 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.706496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fec9c0b-98aa-414b-abe5-9483d44f32a4-console-oauth-config\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.706870 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.706853    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fec9c0b-98aa-414b-abe5-9483d44f32a4-console-serving-cert\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.713003 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.712971    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgtcz\" (UniqueName: \"kubernetes.io/projected/0fec9c0b-98aa-414b-abe5-9483d44f32a4-kube-api-access-zgtcz\") pod \"console-688bf79bb6-8v5w4\" (UID: \"0fec9c0b-98aa-414b-abe5-9483d44f32a4\") " pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.831966 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.831878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:15.836306 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.836275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn" event={"ID":"998fe068-bc7f-4aae-8f7c-d87e468c5db8","Type":"ContainerStarted","Data":"a350ea97e993ed48e9777d08dfad4a2855ba30699303b49f16fe4635f4093add"}
Sep 12 01:51:15.838060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.838036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w" event={"ID":"6d2c421b-3698-49dc-a82e-75e7be8c109f","Type":"ContainerStarted","Data":"173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f"}
Sep 12 01:51:15.857727 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.857677    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn" podStartSLOduration=2.306998382 podStartE2EDuration="4.857660495s" totalImagesPullingTime="2.550662113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:51:15.856552373 +0000 UTC m=+381.383453448" watchObservedRunningTime="2026-09-12 01:51:15.857660495 +0000 UTC m=+381.384561575"
Sep 12 01:51:15.878532 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:15.878459    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w" podStartSLOduration=2.054540223 podStartE2EDuration="4.878435503s" totalImagesPullingTime="2.82389528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:51:15.877460086 +0000 UTC m=+381.404361173" watchObservedRunningTime="2026-09-12 01:51:15.878435503 +0000 UTC m=+381.405336585"
Sep 12 01:51:16.061760 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.061722    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:16.066571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.066542    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:16.124455 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.124423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688bf79bb6-8v5w4"]
Sep 12 01:51:16.126649 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:51:16.126620    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fec9c0b_98aa_414b_abe5_9483d44f32a4.slice/crio-e2b13953b9722c57753cfa83b7dcf9f1af366166075829f301595ec4e9ef68b0 WatchSource:0}: Error finding container e2b13953b9722c57753cfa83b7dcf9f1af366166075829f301595ec4e9ef68b0: Status 404 returned error can't find the container with id e2b13953b9722c57753cfa83b7dcf9f1af366166075829f301595ec4e9ef68b0
Sep 12 01:51:16.781058 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.781022    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:16.782689 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.782658    2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 12 01:51:16.782838 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.782722    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w" podUID="6d2c421b-3698-49dc-a82e-75e7be8c109f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 12 01:51:16.845041 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.844996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688bf79bb6-8v5w4" event={"ID":"0fec9c0b-98aa-414b-abe5-9483d44f32a4","Type":"ContainerStarted","Data":"7dad2da55f8d57519cd8390b7bbc32c6fc981a35a6d43205fec2418e8fb83cad"}
Sep 12 01:51:16.845242 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.845139    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:16.845242 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.845200    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688bf79bb6-8v5w4" event={"ID":"0fec9c0b-98aa-414b-abe5-9483d44f32a4","Type":"ContainerStarted","Data":"e2b13953b9722c57753cfa83b7dcf9f1af366166075829f301595ec4e9ef68b0"}
Sep 12 01:51:16.846095 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.846076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn"
Sep 12 01:51:16.860991 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.860847    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-688bf79bb6-8v5w4" podStartSLOduration=1.860827612 podStartE2EDuration="1.860827612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:51:16.860813281 +0000 UTC m=+382.387714374" watchObservedRunningTime="2026-09-12 01:51:16.860827612 +0000 UTC m=+382.387728689"
Sep 12 01:51:16.906280 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:16.906237    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"]
Sep 12 01:51:17.780880 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:17.780839    2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 12 01:51:17.781266 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:17.780905    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w" podUID="6d2c421b-3698-49dc-a82e-75e7be8c109f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 12 01:51:18.780990 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:18.780947    2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 12 01:51:18.781396 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:18.781020    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w" podUID="6d2c421b-3698-49dc-a82e-75e7be8c109f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 12 01:51:18.792239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:18.792198    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-b9f9494c8-njtqr"
Sep 12 01:51:18.854695 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:18.854650    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w" podUID="6d2c421b-3698-49dc-a82e-75e7be8c109f" containerName="istio-proxy" containerID="cri-o://173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f" gracePeriod=30
Sep 12 01:51:24.101571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.101544    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:24.185731 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.185644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c421b-3698-49dc-a82e-75e7be8c109f-istiod-ca-cert\") pod \"6d2c421b-3698-49dc-a82e-75e7be8c109f\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") "
Sep 12 01:51:24.185731 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.185699    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-certs\") pod \"6d2c421b-3698-49dc-a82e-75e7be8c109f\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") "
Sep 12 01:51:24.185924 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.185768    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-socket\") pod \"6d2c421b-3698-49dc-a82e-75e7be8c109f\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") "
Sep 12 01:51:24.185924 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.185817    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-credential-socket\") pod \"6d2c421b-3698-49dc-a82e-75e7be8c109f\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") "
Sep 12 01:51:24.185924 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.185848    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bchj7\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-kube-api-access-bchj7\") pod \"6d2c421b-3698-49dc-a82e-75e7be8c109f\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") "
Sep 12 01:51:24.185924 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.185894    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-data\") pod \"6d2c421b-3698-49dc-a82e-75e7be8c109f\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") "
Sep 12 01:51:24.186107 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.185946    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-envoy\") pod \"6d2c421b-3698-49dc-a82e-75e7be8c109f\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") "
Sep 12 01:51:24.186107 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.185987    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-podinfo\") pod \"6d2c421b-3698-49dc-a82e-75e7be8c109f\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") "
Sep 12 01:51:24.186107 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.185998    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d2c421b-3698-49dc-a82e-75e7be8c109f" (UID: "6d2c421b-3698-49dc-a82e-75e7be8c109f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:51:24.186107 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.186016    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-token\") pod \"6d2c421b-3698-49dc-a82e-75e7be8c109f\" (UID: \"6d2c421b-3698-49dc-a82e-75e7be8c109f\") "
Sep 12 01:51:24.186401 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.186122    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d2c421b-3698-49dc-a82e-75e7be8c109f" (UID: "6d2c421b-3698-49dc-a82e-75e7be8c109f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:51:24.186401 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.186146    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d2c421b-3698-49dc-a82e-75e7be8c109f" (UID: "6d2c421b-3698-49dc-a82e-75e7be8c109f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:51:24.186401 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.186172    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d2c421b-3698-49dc-a82e-75e7be8c109f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d2c421b-3698-49dc-a82e-75e7be8c109f" (UID: "6d2c421b-3698-49dc-a82e-75e7be8c109f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:51:24.186401 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.186304    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c421b-3698-49dc-a82e-75e7be8c109f-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:24.186401 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.186322    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:24.186401 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.186337    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:24.186401 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.186355    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:24.186731 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.186522    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d2c421b-3698-49dc-a82e-75e7be8c109f" (UID: "6d2c421b-3698-49dc-a82e-75e7be8c109f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:51:24.188379 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.188349    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d2c421b-3698-49dc-a82e-75e7be8c109f" (UID: "6d2c421b-3698-49dc-a82e-75e7be8c109f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 01:51:24.188490 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.188395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d2c421b-3698-49dc-a82e-75e7be8c109f" (UID: "6d2c421b-3698-49dc-a82e-75e7be8c109f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:51:24.188615 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.188590    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d2c421b-3698-49dc-a82e-75e7be8c109f" (UID: "6d2c421b-3698-49dc-a82e-75e7be8c109f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:51:24.188678 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.188643    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-kube-api-access-bchj7" (OuterVolumeSpecName: "kube-api-access-bchj7") pod "6d2c421b-3698-49dc-a82e-75e7be8c109f" (UID: "6d2c421b-3698-49dc-a82e-75e7be8c109f"). InnerVolumeSpecName "kube-api-access-bchj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:51:24.287635 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.287590    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:24.287635 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.287627    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bchj7\" (UniqueName: \"kubernetes.io/projected/6d2c421b-3698-49dc-a82e-75e7be8c109f-kube-api-access-bchj7\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:24.287635 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.287637    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:24.287635 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.287647    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:24.287988 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.287658    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c421b-3698-49dc-a82e-75e7be8c109f-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:24.885226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.885175    2580 generic.go:356] "Generic (PLEG): container finished" podID="6d2c421b-3698-49dc-a82e-75e7be8c109f" containerID="173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f" exitCode=0
Sep 12 01:51:24.885432 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.885267    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"
Sep 12 01:51:24.885432 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.885270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w" event={"ID":"6d2c421b-3698-49dc-a82e-75e7be8c109f","Type":"ContainerDied","Data":"173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f"}
Sep 12 01:51:24.885432 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.885342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w" event={"ID":"6d2c421b-3698-49dc-a82e-75e7be8c109f","Type":"ContainerDied","Data":"e73abf51968e87fb00ba6b431a07f00ccd5d5b43bf55d7b0e165da80849090f8"}
Sep 12 01:51:24.885432 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.885386    2580 scope.go:117] "RemoveContainer" containerID="173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f"
Sep 12 01:51:24.894427 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.894405    2580 scope.go:117] "RemoveContainer" containerID="173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f"
Sep 12 01:51:24.894737 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:51:24.894716    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f\": container with ID starting with 173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f not found: ID does not exist" containerID="173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f"
Sep 12 01:51:24.894806 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.894750    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f"} err="failed to get container status \"173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f\": rpc error: code = NotFound desc = could not find container \"173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f\": container with ID starting with 173384e6b0ea4fd453f4f31cdc490ad4d087ec5d4b25c6b03bed03f06eba8a7f not found: ID does not exist"
Sep 12 01:51:24.928362 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.928324    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"]
Sep 12 01:51:24.934426 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.934396    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-jtp4w"]
Sep 12 01:51:24.979472 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:24.979443    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d2c421b-3698-49dc-a82e-75e7be8c109f" path="/var/lib/kubelet/pods/6d2c421b-3698-49dc-a82e-75e7be8c109f/volumes"
Sep 12 01:51:25.675192 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.675156    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"]
Sep 12 01:51:25.675610 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.675479    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d2c421b-3698-49dc-a82e-75e7be8c109f" containerName="istio-proxy"
Sep 12 01:51:25.675610 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.675491    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d2c421b-3698-49dc-a82e-75e7be8c109f" containerName="istio-proxy"
Sep 12 01:51:25.675610 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.675547    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d2c421b-3698-49dc-a82e-75e7be8c109f" containerName="istio-proxy"
Sep 12 01:51:25.687152 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.687094    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:25.689011 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.688979    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"]
Sep 12 01:51:25.689845 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.689824    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6l5jf\""
Sep 12 01:51:25.689948 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.689861    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 12 01:51:25.799981 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.799941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94a96bcc-5538-465b-a756-eca75abcb493-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ffbhv\" (UID: \"94a96bcc-5538-465b-a756-eca75abcb493\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:25.800137 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.800002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc6vr\" (UniqueName: \"kubernetes.io/projected/94a96bcc-5538-465b-a756-eca75abcb493-kube-api-access-wc6vr\") pod \"llmisvc-controller-manager-777c6bd7d5-ffbhv\" (UID: \"94a96bcc-5538-465b-a756-eca75abcb493\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:25.841043 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.841002    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:25.841267 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.841074    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:25.847120 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.847094    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:25.894710 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.894680    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-688bf79bb6-8v5w4"
Sep 12 01:51:25.900395 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.900369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94a96bcc-5538-465b-a756-eca75abcb493-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ffbhv\" (UID: \"94a96bcc-5538-465b-a756-eca75abcb493\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:25.900537 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.900414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc6vr\" (UniqueName: \"kubernetes.io/projected/94a96bcc-5538-465b-a756-eca75abcb493-kube-api-access-wc6vr\") pod \"llmisvc-controller-manager-777c6bd7d5-ffbhv\" (UID: \"94a96bcc-5538-465b-a756-eca75abcb493\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:25.900600 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:51:25.900533    2580 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 12 01:51:25.900669 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:51:25.900623    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94a96bcc-5538-465b-a756-eca75abcb493-cert podName:94a96bcc-5538-465b-a756-eca75abcb493 nodeName:}" failed. No retries permitted until 2026-09-12 01:51:26.40060103 +0000 UTC m=+391.927502102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/94a96bcc-5538-465b-a756-eca75abcb493-cert") pod "llmisvc-controller-manager-777c6bd7d5-ffbhv" (UID: "94a96bcc-5538-465b-a756-eca75abcb493") : secret "llmisvc-webhook-server-cert" not found
Sep 12 01:51:25.927381 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:25.927302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc6vr\" (UniqueName: \"kubernetes.io/projected/94a96bcc-5538-465b-a756-eca75abcb493-kube-api-access-wc6vr\") pod \"llmisvc-controller-manager-777c6bd7d5-ffbhv\" (UID: \"94a96bcc-5538-465b-a756-eca75abcb493\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:26.022073 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:26.022037    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78f8d6fb4-nx6m6"]
Sep 12 01:51:26.405077 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:26.405039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94a96bcc-5538-465b-a756-eca75abcb493-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ffbhv\" (UID: \"94a96bcc-5538-465b-a756-eca75abcb493\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:26.407490 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:26.407463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94a96bcc-5538-465b-a756-eca75abcb493-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ffbhv\" (UID: \"94a96bcc-5538-465b-a756-eca75abcb493\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:26.597139 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:26.597102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:26.874558 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:26.874528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"]
Sep 12 01:51:26.876783 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:51:26.876752    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod94a96bcc_5538_465b_a756_eca75abcb493.slice/crio-c6cf41d553b6847608e428d4aeeba1982af2cedbe33c6685f19d773e227e3b67 WatchSource:0}: Error finding container c6cf41d553b6847608e428d4aeeba1982af2cedbe33c6685f19d773e227e3b67: Status 404 returned error can't find the container with id c6cf41d553b6847608e428d4aeeba1982af2cedbe33c6685f19d773e227e3b67
Sep 12 01:51:26.897238 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:26.897181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv" event={"ID":"94a96bcc-5538-465b-a756-eca75abcb493","Type":"ContainerStarted","Data":"c6cf41d553b6847608e428d4aeeba1982af2cedbe33c6685f19d773e227e3b67"}
Sep 12 01:51:29.917099 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:29.917058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv" event={"ID":"94a96bcc-5538-465b-a756-eca75abcb493","Type":"ContainerStarted","Data":"dfffb8e15ae71a0e4a9ad4fa0107c8e295759fc9ab63c00c7794d13552c9da69"}
Sep 12 01:51:29.917534 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:29.917248    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:51:29.933602 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:29.933548    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv" podStartSLOduration=2.462215778 podStartE2EDuration="4.933530994s" totalImagesPullingTime="2.471315216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:51:29.932146464 +0000 UTC m=+395.459047546" watchObservedRunningTime="2026-09-12 01:51:29.933530994 +0000 UTC m=+395.460432074"
Sep 12 01:51:51.043585 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.043546    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78f8d6fb4-nx6m6" podUID="e25e8f41-1278-4e4a-ad36-2f47867ca23e" containerName="console" containerID="cri-o://084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037" gracePeriod=15
Sep 12 01:51:51.288443 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.288418    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:51:51.326092 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326007    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-config\") pod \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") "
Sep 12 01:51:51.326092 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326071    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-oauth-serving-cert\") pod \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") "
Sep 12 01:51:51.326323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326102    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-trusted-ca-bundle\") pod \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") "
Sep 12 01:51:51.326323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326165    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-serving-cert\") pod \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") "
Sep 12 01:51:51.326323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326223    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-oauth-config\") pod \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") "
Sep 12 01:51:51.326323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326269    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmcbc\" (UniqueName: \"kubernetes.io/projected/e25e8f41-1278-4e4a-ad36-2f47867ca23e-kube-api-access-bmcbc\") pod \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") "
Sep 12 01:51:51.326323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-service-ca\") pod \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\" (UID: \"e25e8f41-1278-4e4a-ad36-2f47867ca23e\") "
Sep 12 01:51:51.326595 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326567    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-config" (OuterVolumeSpecName: "console-config") pod "e25e8f41-1278-4e4a-ad36-2f47867ca23e" (UID: "e25e8f41-1278-4e4a-ad36-2f47867ca23e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:51:51.326683 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326648    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e25e8f41-1278-4e4a-ad36-2f47867ca23e" (UID: "e25e8f41-1278-4e4a-ad36-2f47867ca23e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:51:51.326851 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326782    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-service-ca" (OuterVolumeSpecName: "service-ca") pod "e25e8f41-1278-4e4a-ad36-2f47867ca23e" (UID: "e25e8f41-1278-4e4a-ad36-2f47867ca23e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:51:51.326851 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326846    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e25e8f41-1278-4e4a-ad36-2f47867ca23e" (UID: "e25e8f41-1278-4e4a-ad36-2f47867ca23e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:51:51.327049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326894    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-trusted-ca-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:51.327049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326916    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:51.327049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.326930    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-config\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:51.328631 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.328606    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e25e8f41-1278-4e4a-ad36-2f47867ca23e" (UID: "e25e8f41-1278-4e4a-ad36-2f47867ca23e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:51:51.329036 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.329013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e25e8f41-1278-4e4a-ad36-2f47867ca23e" (UID: "e25e8f41-1278-4e4a-ad36-2f47867ca23e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:51:51.329036 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.329013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25e8f41-1278-4e4a-ad36-2f47867ca23e-kube-api-access-bmcbc" (OuterVolumeSpecName: "kube-api-access-bmcbc") pod "e25e8f41-1278-4e4a-ad36-2f47867ca23e" (UID: "e25e8f41-1278-4e4a-ad36-2f47867ca23e"). InnerVolumeSpecName "kube-api-access-bmcbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:51:51.428311 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.428278    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-serving-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:51.428311 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.428304    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e25e8f41-1278-4e4a-ad36-2f47867ca23e-console-oauth-config\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:51.428311 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.428314    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmcbc\" (UniqueName: \"kubernetes.io/projected/e25e8f41-1278-4e4a-ad36-2f47867ca23e-kube-api-access-bmcbc\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:51.428511 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:51.428323    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e25e8f41-1278-4e4a-ad36-2f47867ca23e-oauth-serving-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:51:52.021956 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.021919    2580 generic.go:356] "Generic (PLEG): container finished" podID="e25e8f41-1278-4e4a-ad36-2f47867ca23e" containerID="084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037" exitCode=0
Sep 12 01:51:52.021956 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.021959    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f8d6fb4-nx6m6"
Sep 12 01:51:52.022181 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.022006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f8d6fb4-nx6m6" event={"ID":"e25e8f41-1278-4e4a-ad36-2f47867ca23e","Type":"ContainerDied","Data":"084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037"}
Sep 12 01:51:52.022181 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.022053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f8d6fb4-nx6m6" event={"ID":"e25e8f41-1278-4e4a-ad36-2f47867ca23e","Type":"ContainerDied","Data":"df6bddef505b17b074df3735130c1e8a871c38e188f816411a4812161ef34c7b"}
Sep 12 01:51:52.022181 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.022078    2580 scope.go:117] "RemoveContainer" containerID="084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037"
Sep 12 01:51:52.030904 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.030887    2580 scope.go:117] "RemoveContainer" containerID="084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037"
Sep 12 01:51:52.031165 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:51:52.031143    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037\": container with ID starting with 084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037 not found: ID does not exist" containerID="084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037"
Sep 12 01:51:52.031278 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.031171    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037"} err="failed to get container status \"084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037\": rpc error: code = NotFound desc = could not find container \"084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037\": container with ID starting with 084f618ae9b45517351fe0ca1851d5cca10b90bdcaa93f078ecab4a934be5037 not found: ID does not exist"
Sep 12 01:51:52.043808 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.043782    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78f8d6fb4-nx6m6"]
Sep 12 01:51:52.046181 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.046161    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78f8d6fb4-nx6m6"]
Sep 12 01:51:52.978746 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:51:52.978714    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e25e8f41-1278-4e4a-ad36-2f47867ca23e" path="/var/lib/kubelet/pods/e25e8f41-1278-4e4a-ad36-2f47867ca23e/volumes"
Sep 12 01:52:00.929123 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:00.929095    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffbhv"
Sep 12 01:52:22.859906 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.859824    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"]
Sep 12 01:52:22.860385 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.860154    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e25e8f41-1278-4e4a-ad36-2f47867ca23e" containerName="console"
Sep 12 01:52:22.860385 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.860164    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e25e8f41-1278-4e4a-ad36-2f47867ca23e" containerName="console"
Sep 12 01:52:22.860385 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.860235    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e25e8f41-1278-4e4a-ad36-2f47867ca23e" containerName="console"
Sep 12 01:52:22.863327 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.863310    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:22.866030 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.866007    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 12 01:52:22.866030 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.866025    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 12 01:52:22.866176 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.866088    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-txj2m\""
Sep 12 01:52:22.871767 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.871746    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"]
Sep 12 01:52:22.981080 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.981041    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"]
Sep 12 01:52:22.984498 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.984482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:22.996219 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.996181    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"]
Sep 12 01:52:22.997393 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.997371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:22.997529 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.997417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbtbw\" (UniqueName: \"kubernetes.io/projected/129851dd-0f12-42be-a7cf-3133db7bda69-kube-api-access-lbtbw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:22.997592 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:22.997524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:23.068239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.068196    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"]
Sep 12 01:52:23.071786 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.071763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.082552 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.082528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"]
Sep 12 01:52:23.098682 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.098656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:23.098797 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.098730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.098797 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.098758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:23.098797 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.098786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.098927 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.098812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5svf\" (UniqueName: \"kubernetes.io/projected/6d9cff24-e405-4f93-bc57-35e089420afc-kube-api-access-s5svf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.098969 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.098923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbtbw\" (UniqueName: \"kubernetes.io/projected/129851dd-0f12-42be-a7cf-3133db7bda69-kube-api-access-lbtbw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:23.099060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.099046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:23.099108 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.099083    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:23.114839 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.114777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbtbw\" (UniqueName: \"kubernetes.io/projected/129851dd-0f12-42be-a7cf-3133db7bda69-kube-api-access-lbtbw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:23.172870 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.172842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:23.175388 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.175362    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"]
Sep 12 01:52:23.178592 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.178574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.193356 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.193334    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"]
Sep 12 01:52:23.199547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.199508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.199664 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.199574    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pljq4\" (UniqueName: \"kubernetes.io/projected/1a46334c-bce0-41fb-ba88-2a122c53bc9e-kube-api-access-pljq4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.199664 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.199614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.199781 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.199688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.199781 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.199731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.199781 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.199757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5svf\" (UniqueName: \"kubernetes.io/projected/6d9cff24-e405-4f93-bc57-35e089420afc-kube-api-access-s5svf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.200166 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.200147    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.200240 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.200175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.209736 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.209708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5svf\" (UniqueName: \"kubernetes.io/projected/6d9cff24-e405-4f93-bc57-35e089420afc-kube-api-access-s5svf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.293595 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.293562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:23.300762 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.300741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.300833 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.300793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.300833 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.300823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.300929 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.300848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pljq4\" (UniqueName: \"kubernetes.io/projected/1a46334c-bce0-41fb-ba88-2a122c53bc9e-kube-api-access-pljq4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.300929 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.300868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.300929 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.300897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m9gb\" (UniqueName: \"kubernetes.io/projected/a7af0db5-bb0d-4600-a969-c5be7d829635-kube-api-access-8m9gb\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.301226 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.301192    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.301294 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.301231    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.312017 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.311995    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pljq4\" (UniqueName: \"kubernetes.io/projected/1a46334c-bce0-41fb-ba88-2a122c53bc9e-kube-api-access-pljq4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.380874 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.380796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:23.402185 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.402153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.402315 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.402260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m9gb\" (UniqueName: \"kubernetes.io/projected/a7af0db5-bb0d-4600-a969-c5be7d829635-kube-api-access-8m9gb\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.402375 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.402324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.402543 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.402522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.402614 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.402588    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.411466 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.411443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m9gb\" (UniqueName: \"kubernetes.io/projected/a7af0db5-bb0d-4600-a969-c5be7d829635-kube-api-access-8m9gb\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.450761 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.450732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"]
Sep 12 01:52:23.451907 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:52:23.451881    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod129851dd_0f12_42be_a7cf_3133db7bda69.slice/crio-731665a07199fd8ffc8e911a2f0a542f284757c14536050bf454c51457cccb6c WatchSource:0}: Error finding container 731665a07199fd8ffc8e911a2f0a542f284757c14536050bf454c51457cccb6c: Status 404 returned error can't find the container with id 731665a07199fd8ffc8e911a2f0a542f284757c14536050bf454c51457cccb6c
Sep 12 01:52:23.501684 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.501656    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:23.571997 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.571965    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"]
Sep 12 01:52:23.595330 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:52:23.595298    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9cff24_e405_4f93_bc57_35e089420afc.slice/crio-a9f852c12b05c13cf5bd8c5c59fd0a5362ba1a7b0aab5a5d7bb071b13c8759eb WatchSource:0}: Error finding container a9f852c12b05c13cf5bd8c5c59fd0a5362ba1a7b0aab5a5d7bb071b13c8759eb: Status 404 returned error can't find the container with id a9f852c12b05c13cf5bd8c5c59fd0a5362ba1a7b0aab5a5d7bb071b13c8759eb
Sep 12 01:52:23.653166 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.653140    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"]
Sep 12 01:52:23.654965 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:52:23.654934    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a46334c_bce0_41fb_ba88_2a122c53bc9e.slice/crio-20c8ddfe2a3975271d606ea77849beafe56ba0a469089ebb1cae81d60e9f0cef WatchSource:0}: Error finding container 20c8ddfe2a3975271d606ea77849beafe56ba0a469089ebb1cae81d60e9f0cef: Status 404 returned error can't find the container with id 20c8ddfe2a3975271d606ea77849beafe56ba0a469089ebb1cae81d60e9f0cef
Sep 12 01:52:23.774931 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:23.774903    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"]
Sep 12 01:52:23.775880 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:52:23.775859    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7af0db5_bb0d_4600_a969_c5be7d829635.slice/crio-bb41c36608fe01fa9980c2ea9c6497433a7465a278cd833d6a1431775b3fb103 WatchSource:0}: Error finding container bb41c36608fe01fa9980c2ea9c6497433a7465a278cd833d6a1431775b3fb103: Status 404 returned error can't find the container with id bb41c36608fe01fa9980c2ea9c6497433a7465a278cd833d6a1431775b3fb103
Sep 12 01:52:24.177583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.177489    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerID="efbd85922219dd985438d5849309d75d1873a99faccef19a865a9fb3259115ef" exitCode=0
Sep 12 01:52:24.177583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.177574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq" event={"ID":"1a46334c-bce0-41fb-ba88-2a122c53bc9e","Type":"ContainerDied","Data":"efbd85922219dd985438d5849309d75d1873a99faccef19a865a9fb3259115ef"}
Sep 12 01:52:24.178070 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.177622    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq" event={"ID":"1a46334c-bce0-41fb-ba88-2a122c53bc9e","Type":"ContainerStarted","Data":"20c8ddfe2a3975271d606ea77849beafe56ba0a469089ebb1cae81d60e9f0cef"}
Sep 12 01:52:24.179263 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.179204    2580 generic.go:356] "Generic (PLEG): container finished" podID="6d9cff24-e405-4f93-bc57-35e089420afc" containerID="cf98bf829eddec373d17d1896dea2c8929f5d313606f2f9cbd1f633b92e68a32" exitCode=0
Sep 12 01:52:24.179345 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.179315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42" event={"ID":"6d9cff24-e405-4f93-bc57-35e089420afc","Type":"ContainerDied","Data":"cf98bf829eddec373d17d1896dea2c8929f5d313606f2f9cbd1f633b92e68a32"}
Sep 12 01:52:24.179399 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.179366    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42" event={"ID":"6d9cff24-e405-4f93-bc57-35e089420afc","Type":"ContainerStarted","Data":"a9f852c12b05c13cf5bd8c5c59fd0a5362ba1a7b0aab5a5d7bb071b13c8759eb"}
Sep 12 01:52:24.181139 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.181119    2580 generic.go:356] "Generic (PLEG): container finished" podID="129851dd-0f12-42be-a7cf-3133db7bda69" containerID="a709bad3fdb62ae1628be569ddeb790292f75e4c116434ae4ebae82e57b4e8b4" exitCode=0
Sep 12 01:52:24.181271 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.181205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz" event={"ID":"129851dd-0f12-42be-a7cf-3133db7bda69","Type":"ContainerDied","Data":"a709bad3fdb62ae1628be569ddeb790292f75e4c116434ae4ebae82e57b4e8b4"}
Sep 12 01:52:24.181350 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.181284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz" event={"ID":"129851dd-0f12-42be-a7cf-3133db7bda69","Type":"ContainerStarted","Data":"731665a07199fd8ffc8e911a2f0a542f284757c14536050bf454c51457cccb6c"}
Sep 12 01:52:24.182955 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.182932    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerID="0569c81708b872c8ddac90ee65cccd5d142ae3accf7ba88c36c7920c8255f7df" exitCode=0
Sep 12 01:52:24.183027 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.182976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq" event={"ID":"a7af0db5-bb0d-4600-a969-c5be7d829635","Type":"ContainerDied","Data":"0569c81708b872c8ddac90ee65cccd5d142ae3accf7ba88c36c7920c8255f7df"}
Sep 12 01:52:24.183027 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:24.183011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq" event={"ID":"a7af0db5-bb0d-4600-a969-c5be7d829635","Type":"ContainerStarted","Data":"bb41c36608fe01fa9980c2ea9c6497433a7465a278cd833d6a1431775b3fb103"}
Sep 12 01:52:25.190263 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:25.190233    2580 generic.go:356] "Generic (PLEG): container finished" podID="6d9cff24-e405-4f93-bc57-35e089420afc" containerID="ed7096ca0baedd318392e0fd9a3f62c57e568435cc9aa18ec1c4f1f4fa524118" exitCode=0
Sep 12 01:52:25.190623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:25.190253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42" event={"ID":"6d9cff24-e405-4f93-bc57-35e089420afc","Type":"ContainerDied","Data":"ed7096ca0baedd318392e0fd9a3f62c57e568435cc9aa18ec1c4f1f4fa524118"}
Sep 12 01:52:26.197571 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:26.197540    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerID="d211df97284c8adba9c212310117f76332f9c5ec02c54a39eb5facfc8a417076" exitCode=0
Sep 12 01:52:26.198003 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:26.197631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq" event={"ID":"a7af0db5-bb0d-4600-a969-c5be7d829635","Type":"ContainerDied","Data":"d211df97284c8adba9c212310117f76332f9c5ec02c54a39eb5facfc8a417076"}
Sep 12 01:52:26.199711 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:26.199689    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerID="abec1eba1bc4601682d025e1addeb2b980c73f90df8aefe9eb3a60991401e2df" exitCode=0
Sep 12 01:52:26.199815 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:26.199735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq" event={"ID":"1a46334c-bce0-41fb-ba88-2a122c53bc9e","Type":"ContainerDied","Data":"abec1eba1bc4601682d025e1addeb2b980c73f90df8aefe9eb3a60991401e2df"}
Sep 12 01:52:26.202276 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:26.202254    2580 generic.go:356] "Generic (PLEG): container finished" podID="6d9cff24-e405-4f93-bc57-35e089420afc" containerID="c50804daa58afed590dba1e60227c00df26acb948fb8aaf458d99bb9d4021c1d" exitCode=0
Sep 12 01:52:26.202363 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:26.202326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42" event={"ID":"6d9cff24-e405-4f93-bc57-35e089420afc","Type":"ContainerDied","Data":"c50804daa58afed590dba1e60227c00df26acb948fb8aaf458d99bb9d4021c1d"}
Sep 12 01:52:26.204164 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:26.204140    2580 generic.go:356] "Generic (PLEG): container finished" podID="129851dd-0f12-42be-a7cf-3133db7bda69" containerID="1906db88471183d9a5c434b08dc7d4040c5f7aeef86f40b9eb99f7d906840f4c" exitCode=0
Sep 12 01:52:26.204264 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:26.204184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz" event={"ID":"129851dd-0f12-42be-a7cf-3133db7bda69","Type":"ContainerDied","Data":"1906db88471183d9a5c434b08dc7d4040c5f7aeef86f40b9eb99f7d906840f4c"}
Sep 12 01:52:27.217503 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.217471    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerID="15dec357fcc0c6848de505666d34a33926b313602e104c4565635d0ecb5a225a" exitCode=0
Sep 12 01:52:27.217913 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.217547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq" event={"ID":"1a46334c-bce0-41fb-ba88-2a122c53bc9e","Type":"ContainerDied","Data":"15dec357fcc0c6848de505666d34a33926b313602e104c4565635d0ecb5a225a"}
Sep 12 01:52:27.219630 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.219605    2580 generic.go:356] "Generic (PLEG): container finished" podID="129851dd-0f12-42be-a7cf-3133db7bda69" containerID="19df3909ce4b09708369ada36294fb6cf96a8b53264167e36dc7db8d8741bd75" exitCode=0
Sep 12 01:52:27.219767 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.219686    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz" event={"ID":"129851dd-0f12-42be-a7cf-3133db7bda69","Type":"ContainerDied","Data":"19df3909ce4b09708369ada36294fb6cf96a8b53264167e36dc7db8d8741bd75"}
Sep 12 01:52:27.221730 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.221709    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerID="73d9d1ce1861cf8a1cd530dc55999be2207b32e7a71136210d7ab478b2016042" exitCode=0
Sep 12 01:52:27.221845 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.221787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq" event={"ID":"a7af0db5-bb0d-4600-a969-c5be7d829635","Type":"ContainerDied","Data":"73d9d1ce1861cf8a1cd530dc55999be2207b32e7a71136210d7ab478b2016042"}
Sep 12 01:52:27.350337 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.350312    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:27.440438 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.440343    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-util\") pod \"6d9cff24-e405-4f93-bc57-35e089420afc\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") "
Sep 12 01:52:27.440591 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.440453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-bundle\") pod \"6d9cff24-e405-4f93-bc57-35e089420afc\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") "
Sep 12 01:52:27.440591 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.440496    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5svf\" (UniqueName: \"kubernetes.io/projected/6d9cff24-e405-4f93-bc57-35e089420afc-kube-api-access-s5svf\") pod \"6d9cff24-e405-4f93-bc57-35e089420afc\" (UID: \"6d9cff24-e405-4f93-bc57-35e089420afc\") "
Sep 12 01:52:27.440890 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.440859    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-bundle" (OuterVolumeSpecName: "bundle") pod "6d9cff24-e405-4f93-bc57-35e089420afc" (UID: "6d9cff24-e405-4f93-bc57-35e089420afc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:52:27.442669 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.442646    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d9cff24-e405-4f93-bc57-35e089420afc-kube-api-access-s5svf" (OuterVolumeSpecName: "kube-api-access-s5svf") pod "6d9cff24-e405-4f93-bc57-35e089420afc" (UID: "6d9cff24-e405-4f93-bc57-35e089420afc"). InnerVolumeSpecName "kube-api-access-s5svf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:52:27.445964 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.445941    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-util" (OuterVolumeSpecName: "util") pod "6d9cff24-e405-4f93-bc57-35e089420afc" (UID: "6d9cff24-e405-4f93-bc57-35e089420afc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:52:27.541452 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.541422    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:27.541452 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.541447    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5svf\" (UniqueName: \"kubernetes.io/projected/6d9cff24-e405-4f93-bc57-35e089420afc-kube-api-access-s5svf\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:27.541452 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:27.541463    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cff24-e405-4f93-bc57-35e089420afc-util\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:28.228483 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.228448    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42"
Sep 12 01:52:28.228483 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.228469    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wzr42" event={"ID":"6d9cff24-e405-4f93-bc57-35e089420afc","Type":"ContainerDied","Data":"a9f852c12b05c13cf5bd8c5c59fd0a5362ba1a7b0aab5a5d7bb071b13c8759eb"}
Sep 12 01:52:28.228885 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.228517    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9f852c12b05c13cf5bd8c5c59fd0a5362ba1a7b0aab5a5d7bb071b13c8759eb"
Sep 12 01:52:28.368495 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.368466    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:28.416064 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.416041    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:28.420370 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.420349    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:28.550761 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.550729    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-util\") pod \"129851dd-0f12-42be-a7cf-3133db7bda69\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") "
Sep 12 01:52:28.551056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.550778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbtbw\" (UniqueName: \"kubernetes.io/projected/129851dd-0f12-42be-a7cf-3133db7bda69-kube-api-access-lbtbw\") pod \"129851dd-0f12-42be-a7cf-3133db7bda69\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") "
Sep 12 01:52:28.551056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.550804    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-bundle\") pod \"129851dd-0f12-42be-a7cf-3133db7bda69\" (UID: \"129851dd-0f12-42be-a7cf-3133db7bda69\") "
Sep 12 01:52:28.551056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.550840    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m9gb\" (UniqueName: \"kubernetes.io/projected/a7af0db5-bb0d-4600-a969-c5be7d829635-kube-api-access-8m9gb\") pod \"a7af0db5-bb0d-4600-a969-c5be7d829635\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") "
Sep 12 01:52:28.551056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.550901    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-util\") pod \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") "
Sep 12 01:52:28.551056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.550933    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-bundle\") pod \"a7af0db5-bb0d-4600-a969-c5be7d829635\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") "
Sep 12 01:52:28.551056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.550981    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pljq4\" (UniqueName: \"kubernetes.io/projected/1a46334c-bce0-41fb-ba88-2a122c53bc9e-kube-api-access-pljq4\") pod \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") "
Sep 12 01:52:28.551056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.551009    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-util\") pod \"a7af0db5-bb0d-4600-a969-c5be7d829635\" (UID: \"a7af0db5-bb0d-4600-a969-c5be7d829635\") "
Sep 12 01:52:28.551056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.551037    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-bundle\") pod \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\" (UID: \"1a46334c-bce0-41fb-ba88-2a122c53bc9e\") "
Sep 12 01:52:28.551875 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.551683    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-bundle" (OuterVolumeSpecName: "bundle") pod "129851dd-0f12-42be-a7cf-3133db7bda69" (UID: "129851dd-0f12-42be-a7cf-3133db7bda69"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:52:28.551875 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.551816    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-bundle" (OuterVolumeSpecName: "bundle") pod "1a46334c-bce0-41fb-ba88-2a122c53bc9e" (UID: "1a46334c-bce0-41fb-ba88-2a122c53bc9e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:52:28.552109 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.552075    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-bundle" (OuterVolumeSpecName: "bundle") pod "a7af0db5-bb0d-4600-a969-c5be7d829635" (UID: "a7af0db5-bb0d-4600-a969-c5be7d829635"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:52:28.553295 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.553270    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7af0db5-bb0d-4600-a969-c5be7d829635-kube-api-access-8m9gb" (OuterVolumeSpecName: "kube-api-access-8m9gb") pod "a7af0db5-bb0d-4600-a969-c5be7d829635" (UID: "a7af0db5-bb0d-4600-a969-c5be7d829635"). InnerVolumeSpecName "kube-api-access-8m9gb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:52:28.553529 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.553501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/129851dd-0f12-42be-a7cf-3133db7bda69-kube-api-access-lbtbw" (OuterVolumeSpecName: "kube-api-access-lbtbw") pod "129851dd-0f12-42be-a7cf-3133db7bda69" (UID: "129851dd-0f12-42be-a7cf-3133db7bda69"). InnerVolumeSpecName "kube-api-access-lbtbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:52:28.553948 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.553924    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a46334c-bce0-41fb-ba88-2a122c53bc9e-kube-api-access-pljq4" (OuterVolumeSpecName: "kube-api-access-pljq4") pod "1a46334c-bce0-41fb-ba88-2a122c53bc9e" (UID: "1a46334c-bce0-41fb-ba88-2a122c53bc9e"). InnerVolumeSpecName "kube-api-access-pljq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:52:28.556411 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.556387    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-util" (OuterVolumeSpecName: "util") pod "129851dd-0f12-42be-a7cf-3133db7bda69" (UID: "129851dd-0f12-42be-a7cf-3133db7bda69"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:52:28.560279 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.560255    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-util" (OuterVolumeSpecName: "util") pod "1a46334c-bce0-41fb-ba88-2a122c53bc9e" (UID: "1a46334c-bce0-41fb-ba88-2a122c53bc9e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:52:28.561089 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.561071    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-util" (OuterVolumeSpecName: "util") pod "a7af0db5-bb0d-4600-a969-c5be7d829635" (UID: "a7af0db5-bb0d-4600-a969-c5be7d829635"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:52:28.652583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.652532    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m9gb\" (UniqueName: \"kubernetes.io/projected/a7af0db5-bb0d-4600-a969-c5be7d829635-kube-api-access-8m9gb\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:28.652583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.652578    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-util\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:28.652583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.652587    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:28.652583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.652595    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pljq4\" (UniqueName: \"kubernetes.io/projected/1a46334c-bce0-41fb-ba88-2a122c53bc9e-kube-api-access-pljq4\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:28.652843 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.652605    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7af0db5-bb0d-4600-a969-c5be7d829635-util\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:28.652843 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.652613    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a46334c-bce0-41fb-ba88-2a122c53bc9e-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:28.652843 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.652624    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-util\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:28.652843 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.652632    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbtbw\" (UniqueName: \"kubernetes.io/projected/129851dd-0f12-42be-a7cf-3133db7bda69-kube-api-access-lbtbw\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:28.652843 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:28.652640    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/129851dd-0f12-42be-a7cf-3133db7bda69-bundle\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:52:29.236305 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:29.236272    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq"
Sep 12 01:52:29.236734 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:29.236275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d777whq" event={"ID":"1a46334c-bce0-41fb-ba88-2a122c53bc9e","Type":"ContainerDied","Data":"20c8ddfe2a3975271d606ea77849beafe56ba0a469089ebb1cae81d60e9f0cef"}
Sep 12 01:52:29.236734 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:29.236397    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20c8ddfe2a3975271d606ea77849beafe56ba0a469089ebb1cae81d60e9f0cef"
Sep 12 01:52:29.238510 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:29.238440    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz"
Sep 12 01:52:29.238510 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:29.238450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cl2lkz" event={"ID":"129851dd-0f12-42be-a7cf-3133db7bda69","Type":"ContainerDied","Data":"731665a07199fd8ffc8e911a2f0a542f284757c14536050bf454c51457cccb6c"}
Sep 12 01:52:29.238510 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:29.238487    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="731665a07199fd8ffc8e911a2f0a542f284757c14536050bf454c51457cccb6c"
Sep 12 01:52:29.240670 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:29.240648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq" event={"ID":"a7af0db5-bb0d-4600-a969-c5be7d829635","Type":"ContainerDied","Data":"bb41c36608fe01fa9980c2ea9c6497433a7465a278cd833d6a1431775b3fb103"}
Sep 12 01:52:29.240670 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:29.240663    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acqlkq"
Sep 12 01:52:29.240804 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:29.240682    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb41c36608fe01fa9980c2ea9c6497433a7465a278cd833d6a1431775b3fb103"
Sep 12 01:52:43.722250 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722199    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"]
Sep 12 01:52:43.722700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722651    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="129851dd-0f12-42be-a7cf-3133db7bda69" containerName="pull"
Sep 12 01:52:43.722700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722668    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="129851dd-0f12-42be-a7cf-3133db7bda69" containerName="pull"
Sep 12 01:52:43.722700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722677    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerName="extract"
Sep 12 01:52:43.722700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722685    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerName="extract"
Sep 12 01:52:43.722700 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722699    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="129851dd-0f12-42be-a7cf-3133db7bda69" containerName="util"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722709    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="129851dd-0f12-42be-a7cf-3133db7bda69" containerName="util"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722719    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerName="util"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722728    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerName="util"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722736    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="129851dd-0f12-42be-a7cf-3133db7bda69" containerName="extract"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722744    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="129851dd-0f12-42be-a7cf-3133db7bda69" containerName="extract"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722760    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerName="extract"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722767    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerName="extract"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722781    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d9cff24-e405-4f93-bc57-35e089420afc" containerName="extract"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722791    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d9cff24-e405-4f93-bc57-35e089420afc" containerName="extract"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722801    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d9cff24-e405-4f93-bc57-35e089420afc" containerName="util"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722809    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d9cff24-e405-4f93-bc57-35e089420afc" containerName="util"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722820    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerName="util"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722828    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerName="util"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722837    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerName="pull"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722845    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerName="pull"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722860    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerName="pull"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722869    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerName="pull"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722881    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d9cff24-e405-4f93-bc57-35e089420afc" containerName="pull"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722889    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d9cff24-e405-4f93-bc57-35e089420afc" containerName="pull"
Sep 12 01:52:43.722967 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722971    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d9cff24-e405-4f93-bc57-35e089420afc" containerName="extract"
Sep 12 01:52:43.723912 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722983    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a46334c-bce0-41fb-ba88-2a122c53bc9e" containerName="extract"
Sep 12 01:52:43.723912 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.722992    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="129851dd-0f12-42be-a7cf-3133db7bda69" containerName="extract"
Sep 12 01:52:43.723912 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.723003    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7af0db5-bb0d-4600-a969-c5be7d829635" containerName="extract"
Sep 12 01:52:43.727651 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.727632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"
Sep 12 01:52:43.731416 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.731389    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 12 01:52:43.731595 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.731570    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 12 01:52:43.732788 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.732772    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 12 01:52:43.734024 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.733999    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-nclqm\""
Sep 12 01:52:43.752124 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.752104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"]
Sep 12 01:52:43.777253 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.777200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwl9q\" (UniqueName: \"kubernetes.io/projected/9603f85b-85f9-430c-a4b1-f6087ae8356e-kube-api-access-jwl9q\") pod \"dns-operator-controller-manager-7b84cd7c9c-g58x8\" (UID: \"9603f85b-85f9-430c-a4b1-f6087ae8356e\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"
Sep 12 01:52:43.877853 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.877823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwl9q\" (UniqueName: \"kubernetes.io/projected/9603f85b-85f9-430c-a4b1-f6087ae8356e-kube-api-access-jwl9q\") pod \"dns-operator-controller-manager-7b84cd7c9c-g58x8\" (UID: \"9603f85b-85f9-430c-a4b1-f6087ae8356e\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"
Sep 12 01:52:43.887307 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:43.887282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwl9q\" (UniqueName: \"kubernetes.io/projected/9603f85b-85f9-430c-a4b1-f6087ae8356e-kube-api-access-jwl9q\") pod \"dns-operator-controller-manager-7b84cd7c9c-g58x8\" (UID: \"9603f85b-85f9-430c-a4b1-f6087ae8356e\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"
Sep 12 01:52:44.037812 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:44.037777    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"
Sep 12 01:52:44.312116 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:44.312084    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"]
Sep 12 01:52:44.314279 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:52:44.314247    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9603f85b_85f9_430c_a4b1_f6087ae8356e.slice/crio-bd55e7dc00f6ff14578da0db543766bdb7cfa618240f5579ea6fdb0a8029a3da WatchSource:0}: Error finding container bd55e7dc00f6ff14578da0db543766bdb7cfa618240f5579ea6fdb0a8029a3da: Status 404 returned error can't find the container with id bd55e7dc00f6ff14578da0db543766bdb7cfa618240f5579ea6fdb0a8029a3da
Sep 12 01:52:45.328013 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:45.327974    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8" event={"ID":"9603f85b-85f9-430c-a4b1-f6087ae8356e","Type":"ContainerStarted","Data":"bd55e7dc00f6ff14578da0db543766bdb7cfa618240f5579ea6fdb0a8029a3da"}
Sep 12 01:52:48.338456 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.338420    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-wdhzq"]
Sep 12 01:52:48.342769 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.342749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq"
Sep 12 01:52:48.345822 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.345796    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vjffc\""
Sep 12 01:52:48.349569 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.349522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8" event={"ID":"9603f85b-85f9-430c-a4b1-f6087ae8356e","Type":"ContainerStarted","Data":"fbbe71fe3f9eea84385bbf12fd4ca4e463b8e5dc5ae896ff37499121c8eb974b"}
Sep 12 01:52:48.349742 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.349597    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"
Sep 12 01:52:48.358496 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.358474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-wdhzq"]
Sep 12 01:52:48.418515 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.418482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk2zr\" (UniqueName: \"kubernetes.io/projected/802f816b-fb4a-4c74-ac0b-0b1d998c94b2-kube-api-access-qk2zr\") pod \"authorino-operator-76b478bd78-wdhzq\" (UID: \"802f816b-fb4a-4c74-ac0b-0b1d998c94b2\") " pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq"
Sep 12 01:52:48.432503 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.432458    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8" podStartSLOduration=2.411976256 podStartE2EDuration="5.432441777s" totalImagesPullingTime="3.020465521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:52:48.431174785 +0000 UTC m=+473.958076066" watchObservedRunningTime="2026-09-12 01:52:48.432441777 +0000 UTC m=+473.959342856"
Sep 12 01:52:48.519530 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.519494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk2zr\" (UniqueName: \"kubernetes.io/projected/802f816b-fb4a-4c74-ac0b-0b1d998c94b2-kube-api-access-qk2zr\") pod \"authorino-operator-76b478bd78-wdhzq\" (UID: \"802f816b-fb4a-4c74-ac0b-0b1d998c94b2\") " pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq"
Sep 12 01:52:48.537969 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.537942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk2zr\" (UniqueName: \"kubernetes.io/projected/802f816b-fb4a-4c74-ac0b-0b1d998c94b2-kube-api-access-qk2zr\") pod \"authorino-operator-76b478bd78-wdhzq\" (UID: \"802f816b-fb4a-4c74-ac0b-0b1d998c94b2\") " pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq"
Sep 12 01:52:48.654322 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.654234    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq"
Sep 12 01:52:48.939421 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:48.939392    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-wdhzq"]
Sep 12 01:52:48.940756 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:52:48.940726    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802f816b_fb4a_4c74_ac0b_0b1d998c94b2.slice/crio-e8f83d8e811fbe0fdbef824e6711f2d5d7056b11a8c270269ee21969b64e5b47 WatchSource:0}: Error finding container e8f83d8e811fbe0fdbef824e6711f2d5d7056b11a8c270269ee21969b64e5b47: Status 404 returned error can't find the container with id e8f83d8e811fbe0fdbef824e6711f2d5d7056b11a8c270269ee21969b64e5b47
Sep 12 01:52:49.355861 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:49.355829    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq" event={"ID":"802f816b-fb4a-4c74-ac0b-0b1d998c94b2","Type":"ContainerStarted","Data":"e8f83d8e811fbe0fdbef824e6711f2d5d7056b11a8c270269ee21969b64e5b47"}
Sep 12 01:52:51.367664 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:51.367626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq" event={"ID":"802f816b-fb4a-4c74-ac0b-0b1d998c94b2","Type":"ContainerStarted","Data":"d111251715e822fe5cbe3fd27f7851223445ddaba374d371031d10ee61b19dfb"}
Sep 12 01:52:51.368128 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:51.367723    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq"
Sep 12 01:52:51.386525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:51.386476    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq" podStartSLOduration=1.637374414 podStartE2EDuration="3.386461733s" totalImagesPullingTime="1.749087319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:52:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:52:51.385568894 +0000 UTC m=+476.912469974" watchObservedRunningTime="2026-09-12 01:52:51.386461733 +0000 UTC m=+476.913362813"
Sep 12 01:52:53.382891 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.382855    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"]
Sep 12 01:52:53.386340 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.386318    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:52:53.391047 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.391026    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-snrcf\""
Sep 12 01:52:53.399589 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.399565    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"]
Sep 12 01:52:53.474105 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.474069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32ef01a0-612e-4d39-aa55-19ca4c7c3335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" (UID: \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:52:53.474285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.474128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsj2d\" (UniqueName: \"kubernetes.io/projected/32ef01a0-612e-4d39-aa55-19ca4c7c3335-kube-api-access-tsj2d\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" (UID: \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:52:53.575437 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.575397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsj2d\" (UniqueName: \"kubernetes.io/projected/32ef01a0-612e-4d39-aa55-19ca4c7c3335-kube-api-access-tsj2d\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" (UID: \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:52:53.575608 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.575476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32ef01a0-612e-4d39-aa55-19ca4c7c3335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" (UID: \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:52:53.575811 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.575796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32ef01a0-612e-4d39-aa55-19ca4c7c3335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" (UID: \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:52:53.586160 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.586137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsj2d\" (UniqueName: \"kubernetes.io/projected/32ef01a0-612e-4d39-aa55-19ca4c7c3335-kube-api-access-tsj2d\") pod \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" (UID: \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:52:53.696471 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.696379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:52:53.981317 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:53.981291    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"]
Sep 12 01:52:53.983770 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:52:53.983739    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ef01a0_612e_4d39_aa55_19ca4c7c3335.slice/crio-7a5815329f6589fd4317ca2b525637995e4ef03d09b4ebc0f5d076e693c22622 WatchSource:0}: Error finding container 7a5815329f6589fd4317ca2b525637995e4ef03d09b4ebc0f5d076e693c22622: Status 404 returned error can't find the container with id 7a5815329f6589fd4317ca2b525637995e4ef03d09b4ebc0f5d076e693c22622
Sep 12 01:52:54.383927 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:54.383891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl" event={"ID":"32ef01a0-612e-4d39-aa55-19ca4c7c3335","Type":"ContainerStarted","Data":"7a5815329f6589fd4317ca2b525637995e4ef03d09b4ebc0f5d076e693c22622"}
Sep 12 01:52:58.411959 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:58.411869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl" event={"ID":"32ef01a0-612e-4d39-aa55-19ca4c7c3335","Type":"ContainerStarted","Data":"21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a"}
Sep 12 01:52:58.411959 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:58.411938    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:52:58.447709 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:58.447659    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl" podStartSLOduration=1.309568084 podStartE2EDuration="5.447645895s" totalImagesPullingTime="4.138077811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:52:58.44736537 +0000 UTC m=+483.974266450" watchObservedRunningTime="2026-09-12 01:52:58.447645895 +0000 UTC m=+483.974547028"
Sep 12 01:52:59.358721 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:52:59.358689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-g58x8"
Sep 12 01:53:01.175673 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.175631    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"]
Sep 12 01:53:01.180688 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.180666    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.184872 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.184852    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 12 01:53:01.184872 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.184866    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 12 01:53:01.185223 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.184964    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bf5qb\""
Sep 12 01:53:01.277387 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.277354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"]
Sep 12 01:53:01.347155 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.347114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mjhv\" (UniqueName: \"kubernetes.io/projected/f995294f-dfa8-4f3d-b6f1-db595e271c9a-kube-api-access-4mjhv\") pod \"kuadrant-console-plugin-fbdd4f445-nrpjf\" (UID: \"f995294f-dfa8-4f3d-b6f1-db595e271c9a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.347155 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.347158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f995294f-dfa8-4f3d-b6f1-db595e271c9a-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-nrpjf\" (UID: \"f995294f-dfa8-4f3d-b6f1-db595e271c9a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.347445 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.347301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f995294f-dfa8-4f3d-b6f1-db595e271c9a-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-nrpjf\" (UID: \"f995294f-dfa8-4f3d-b6f1-db595e271c9a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.448068 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.447983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mjhv\" (UniqueName: \"kubernetes.io/projected/f995294f-dfa8-4f3d-b6f1-db595e271c9a-kube-api-access-4mjhv\") pod \"kuadrant-console-plugin-fbdd4f445-nrpjf\" (UID: \"f995294f-dfa8-4f3d-b6f1-db595e271c9a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.448068 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.448026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f995294f-dfa8-4f3d-b6f1-db595e271c9a-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-nrpjf\" (UID: \"f995294f-dfa8-4f3d-b6f1-db595e271c9a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.448294 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.448089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f995294f-dfa8-4f3d-b6f1-db595e271c9a-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-nrpjf\" (UID: \"f995294f-dfa8-4f3d-b6f1-db595e271c9a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.448806 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.448785    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f995294f-dfa8-4f3d-b6f1-db595e271c9a-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-nrpjf\" (UID: \"f995294f-dfa8-4f3d-b6f1-db595e271c9a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.450445 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.450421    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f995294f-dfa8-4f3d-b6f1-db595e271c9a-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-nrpjf\" (UID: \"f995294f-dfa8-4f3d-b6f1-db595e271c9a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.474679 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.474649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mjhv\" (UniqueName: \"kubernetes.io/projected/f995294f-dfa8-4f3d-b6f1-db595e271c9a-kube-api-access-4mjhv\") pod \"kuadrant-console-plugin-fbdd4f445-nrpjf\" (UID: \"f995294f-dfa8-4f3d-b6f1-db595e271c9a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.489597 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.489574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"
Sep 12 01:53:01.807320 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:01.807292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf"]
Sep 12 01:53:01.810063 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:53:01.810039    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf995294f_dfa8_4f3d_b6f1_db595e271c9a.slice/crio-e082675e4dae4306b5f92db6639108a11d314536e67f2fcb3abb4b77aa0ddc52 WatchSource:0}: Error finding container e082675e4dae4306b5f92db6639108a11d314536e67f2fcb3abb4b77aa0ddc52: Status 404 returned error can't find the container with id e082675e4dae4306b5f92db6639108a11d314536e67f2fcb3abb4b77aa0ddc52
Sep 12 01:53:02.375282 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:02.375245    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-wdhzq"
Sep 12 01:53:02.433768 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:02.433730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf" event={"ID":"f995294f-dfa8-4f3d-b6f1-db595e271c9a","Type":"ContainerStarted","Data":"e082675e4dae4306b5f92db6639108a11d314536e67f2fcb3abb4b77aa0ddc52"}
Sep 12 01:53:07.463691 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:07.463653    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf" event={"ID":"f995294f-dfa8-4f3d-b6f1-db595e271c9a","Type":"ContainerStarted","Data":"b31c852bccf52f53b765c99c748d6b2f76380aa62f5eb6e4d841c2712de54983"}
Sep 12 01:53:07.481755 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:07.481704    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-nrpjf" podStartSLOduration=1.776622201 podStartE2EDuration="6.481689109s" totalImagesPullingTime="4.705066908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:53:07.480543419 +0000 UTC m=+493.007444500" watchObservedRunningTime="2026-09-12 01:53:07.481689109 +0000 UTC m=+493.008590252"
Sep 12 01:53:09.418882 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:09.418853    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:53:11.732370 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.732335    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"]
Sep 12 01:53:11.732752 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.732506    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl" podUID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" containerName="manager" containerID="cri-o://21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a" gracePeriod=2
Sep 12 01:53:11.740662 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.740626    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"]
Sep 12 01:53:11.798076 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.798042    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"]
Sep 12 01:53:11.799034 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.799007    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" containerName="manager"
Sep 12 01:53:11.799034 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.799035    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" containerName="manager"
Sep 12 01:53:11.799256 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.799240    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" containerName="manager"
Sep 12 01:53:11.803476 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.803453    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"
Sep 12 01:53:11.808826 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.808791    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-lbd2b\""
Sep 12 01:53:11.811670 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.811646    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"]
Sep 12 01:53:11.834937 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:53:11.834899    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:53:11.943756 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.943725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjh8b\" (UniqueName: \"kubernetes.io/projected/877279e4-3f88-48dd-82ab-6a423910f54d-kube-api-access-rjh8b\") pod \"limitador-operator-controller-manager-74495c69f8-68wf9\" (UID: \"877279e4-3f88-48dd-82ab-6a423910f54d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"
Sep 12 01:53:11.972492 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:11.972467    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:53:11.974829 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:53:11.974805    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:53:12.045286 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.045252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjh8b\" (UniqueName: \"kubernetes.io/projected/877279e4-3f88-48dd-82ab-6a423910f54d-kube-api-access-rjh8b\") pod \"limitador-operator-controller-manager-74495c69f8-68wf9\" (UID: \"877279e4-3f88-48dd-82ab-6a423910f54d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"
Sep 12 01:53:12.059034 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.059009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjh8b\" (UniqueName: \"kubernetes.io/projected/877279e4-3f88-48dd-82ab-6a423910f54d-kube-api-access-rjh8b\") pod \"limitador-operator-controller-manager-74495c69f8-68wf9\" (UID: \"877279e4-3f88-48dd-82ab-6a423910f54d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"
Sep 12 01:53:12.116344 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.116304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"
Sep 12 01:53:12.146350 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.146323    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32ef01a0-612e-4d39-aa55-19ca4c7c3335-extensions-socket-volume\") pod \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\" (UID: \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\") "
Sep 12 01:53:12.146493 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.146360    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsj2d\" (UniqueName: \"kubernetes.io/projected/32ef01a0-612e-4d39-aa55-19ca4c7c3335-kube-api-access-tsj2d\") pod \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\" (UID: \"32ef01a0-612e-4d39-aa55-19ca4c7c3335\") "
Sep 12 01:53:12.146919 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.146892    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32ef01a0-612e-4d39-aa55-19ca4c7c3335-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "32ef01a0-612e-4d39-aa55-19ca4c7c3335" (UID: "32ef01a0-612e-4d39-aa55-19ca4c7c3335"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:53:12.148605 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.148587    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32ef01a0-612e-4d39-aa55-19ca4c7c3335-kube-api-access-tsj2d" (OuterVolumeSpecName: "kube-api-access-tsj2d") pod "32ef01a0-612e-4d39-aa55-19ca4c7c3335" (UID: "32ef01a0-612e-4d39-aa55-19ca4c7c3335"). InnerVolumeSpecName "kube-api-access-tsj2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:53:12.248097 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.248062    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32ef01a0-612e-4d39-aa55-19ca4c7c3335-extensions-socket-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:53:12.248097 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.248092    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsj2d\" (UniqueName: \"kubernetes.io/projected/32ef01a0-612e-4d39-aa55-19ca4c7c3335-kube-api-access-tsj2d\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:53:12.433173 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.433150    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"]
Sep 12 01:53:12.434853 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:53:12.434824    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877279e4_3f88_48dd_82ab_6a423910f54d.slice/crio-ea3f3b6e4ed54c66c8ebd3db5382bd8d5bb55c72b33bcace7147dda8d7eacf2d WatchSource:0}: Error finding container ea3f3b6e4ed54c66c8ebd3db5382bd8d5bb55c72b33bcace7147dda8d7eacf2d: Status 404 returned error can't find the container with id ea3f3b6e4ed54c66c8ebd3db5382bd8d5bb55c72b33bcace7147dda8d7eacf2d
Sep 12 01:53:12.492335 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.492300    2580 generic.go:356] "Generic (PLEG): container finished" podID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" containerID="21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a" exitCode=0
Sep 12 01:53:12.492430 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.492348    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:53:12.492430 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.492410    2580 scope.go:117] "RemoveContainer" containerID="21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a"
Sep 12 01:53:12.493938 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.493911    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9" event={"ID":"877279e4-3f88-48dd-82ab-6a423910f54d","Type":"ContainerStarted","Data":"ea3f3b6e4ed54c66c8ebd3db5382bd8d5bb55c72b33bcace7147dda8d7eacf2d"}
Sep 12 01:53:12.495376 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:53:12.495346    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:53:12.501962 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.501945    2580 scope.go:117] "RemoveContainer" containerID="21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a"
Sep 12 01:53:12.502243 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:53:12.502222    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a\": container with ID starting with 21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a not found: ID does not exist" containerID="21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a"
Sep 12 01:53:12.502290 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.502249    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a"} err="failed to get container status \"21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a\": rpc error: code = NotFound desc = could not find container \"21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a\": container with ID starting with 21b8df3f5511c9cda5cb3825657fa4c3a62a2664599fcaedea365750d4310f9a not found: ID does not exist"
Sep 12 01:53:12.508314 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:53:12.508290    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-z48fl\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-z48fl"
Sep 12 01:53:12.978448 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:12.978416    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32ef01a0-612e-4d39-aa55-19ca4c7c3335" path="/var/lib/kubelet/pods/32ef01a0-612e-4d39-aa55-19ca4c7c3335/volumes"
Sep 12 01:53:14.508499 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:14.508468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9" event={"ID":"877279e4-3f88-48dd-82ab-6a423910f54d","Type":"ContainerStarted","Data":"28f88fdcb9bc963e742efcbc153a1b5ce25b78dc2ac801fab08a277003e8910c"}
Sep 12 01:53:14.508875 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:14.508597    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"
Sep 12 01:53:14.526477 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:14.526430    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9" podStartSLOduration=1.9515517770000002 podStartE2EDuration="3.52641442s" totalImagesPullingTime="1.574862643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:53:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:53:14.525427447 +0000 UTC m=+500.052328551" watchObservedRunningTime="2026-09-12 01:53:14.52641442 +0000 UTC m=+500.053315500"
Sep 12 01:53:25.516288 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:25.516258    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-68wf9"
Sep 12 01:53:27.883980 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:27.883946    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"]
Sep 12 01:53:27.890424 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:27.890398    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:53:27.893589 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:27.893571    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-snrcf\""
Sep 12 01:53:27.911823 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:27.911794    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"]
Sep 12 01:53:27.955509 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:27.955475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsl29\" (UniqueName: \"kubernetes.io/projected/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-kube-api-access-jsl29\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zz6kc\" (UID: \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:53:27.955654 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:27.955528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zz6kc\" (UID: \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:53:28.056733 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.056693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zz6kc\" (UID: \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:53:28.056915 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.056793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsl29\" (UniqueName: \"kubernetes.io/projected/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-kube-api-access-jsl29\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zz6kc\" (UID: \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:53:28.057056 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.057033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zz6kc\" (UID: \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:53:28.074059 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.074032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsl29\" (UniqueName: \"kubernetes.io/projected/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-kube-api-access-jsl29\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zz6kc\" (UID: \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:53:28.200817 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.200727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:53:28.479753 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.479601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"]
Sep 12 01:53:28.482741 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:53:28.482715    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f915fcb_6af0_4e0e_9294_deb5f7c772dd.slice/crio-2a1647c55def7fa93438a89a59780827c1deee03d5339a0110ba8f8e8cade1e7 WatchSource:0}: Error finding container 2a1647c55def7fa93438a89a59780827c1deee03d5339a0110ba8f8e8cade1e7: Status 404 returned error can't find the container with id 2a1647c55def7fa93438a89a59780827c1deee03d5339a0110ba8f8e8cade1e7
Sep 12 01:53:28.586596 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.586553    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc" event={"ID":"5f915fcb-6af0-4e0e-9294-deb5f7c772dd","Type":"ContainerStarted","Data":"90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9"}
Sep 12 01:53:28.586767 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.586620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc" event={"ID":"5f915fcb-6af0-4e0e-9294-deb5f7c772dd","Type":"ContainerStarted","Data":"2a1647c55def7fa93438a89a59780827c1deee03d5339a0110ba8f8e8cade1e7"}
Sep 12 01:53:28.586767 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.586649    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:53:28.622183 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:28.622137    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc" podStartSLOduration=1.622121687 podStartE2EDuration="1.622121687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:53:28.620309618 +0000 UTC m=+514.147210699" watchObservedRunningTime="2026-09-12 01:53:28.622121687 +0000 UTC m=+514.149022768"
Sep 12 01:53:39.594716 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:53:39.594683    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 01:54:54.927918 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:54:54.927892    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 01:54:54.927918 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:54:54.927903    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 01:56:22.524123 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.524079    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"]
Sep 12 01:56:22.527441 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.527418    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.531290 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.531270    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 12 01:56:22.531290 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.531283    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 12 01:56:22.531460 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.531282    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dzfh7\""
Sep 12 01:56:22.531460 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.531271    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 12 01:56:22.536516 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.536493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"]
Sep 12 01:56:22.695373 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.695327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.695580 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.695387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.695580 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.695465    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.695580 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.695526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mcrb\" (UniqueName: \"kubernetes.io/projected/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-kube-api-access-6mcrb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.695580 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.695566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.695740 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.695585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.797067 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.796980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.797067 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.797030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.797067 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.797049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.797067 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.797068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mcrb\" (UniqueName: \"kubernetes.io/projected/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-kube-api-access-6mcrb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.797436 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.797114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.797436 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.797141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.797557 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.797522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.797612 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.797556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.797612 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.797588    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.799378 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.799361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.799674 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.799659    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.805464 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.805444    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mcrb\" (UniqueName: \"kubernetes.io/projected/f3f4f463-d40d-4ce7-b22c-5fd49621a4f4-kube-api-access-6mcrb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qzgxn\" (UID: \"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:22.838565 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:22.838536    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:23.113607 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:23.113579    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"]
Sep 12 01:56:23.115729 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:56:23.115703    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f4f463_d40d_4ce7_b22c_5fd49621a4f4.slice/crio-70721eb619f3f9fe1177b811967a7a7886329f7223f2601eec2e8d91a53db1cf WatchSource:0}: Error finding container 70721eb619f3f9fe1177b811967a7a7886329f7223f2601eec2e8d91a53db1cf: Status 404 returned error can't find the container with id 70721eb619f3f9fe1177b811967a7a7886329f7223f2601eec2e8d91a53db1cf
Sep 12 01:56:23.117378 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:23.117361    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 01:56:23.518462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:23.518427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn" event={"ID":"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4","Type":"ContainerStarted","Data":"70721eb619f3f9fe1177b811967a7a7886329f7223f2601eec2e8d91a53db1cf"}
Sep 12 01:56:24.532199 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:24.532156    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn" event={"ID":"f3f4f463-d40d-4ce7-b22c-5fd49621a4f4","Type":"ContainerStarted","Data":"3fa0e35ec7c5f3d18ec7e0374085d3430c739bbd36361b213ecabb942b2066db"}
Sep 12 01:56:24.552415 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:24.552360    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn" podStartSLOduration=1.588268928 podStartE2EDuration="2.552343898s" totalImagesPullingTime="964.07497ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:56:24.550824335 +0000 UTC m=+690.077725415" watchObservedRunningTime="2026-09-12 01:56:24.552343898 +0000 UTC m=+690.079244977"
Sep 12 01:56:32.839242 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:32.839184    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:32.839668 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:32.839297    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:32.851643 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:32.851622    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:56:33.593897 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:56:33.593860    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qzgxn"
Sep 12 01:58:03.571923 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.571877    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"]
Sep 12 01:58:03.576106 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.576082    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.579224 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.579148    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-98f68e-openshift-default-dockercfg-2xrlf\""
Sep 12 01:58:03.591429 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.591400    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"]
Sep 12 01:58:03.654590 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.654553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-socket\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.654590 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.654587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-token\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.654783 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.654609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istiod-ca-cert\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.654783 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.654665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5x5t\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-kube-api-access-p5x5t\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.654783 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.654716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-podinfo\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.654783 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.654742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-credential-socket\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.654915 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.654811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-envoy\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.654915 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.654831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-data\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.654915 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.654848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-certs\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.755393 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-socket\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.755393 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-token\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.755662 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istiod-ca-cert\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.755662 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5x5t\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-kube-api-access-p5x5t\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.755662 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-podinfo\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.755662 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-credential-socket\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.755662 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-envoy\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.755662 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-data\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.755662 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-certs\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.756013 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.755802    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-socket\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.756072 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.756042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-data\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.756129 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.756084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-credential-socket\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.756178 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.756152    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-certs\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.756713 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.756685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istiod-ca-cert\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.758940 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.758913    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-envoy\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.759093 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.759056    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-podinfo\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.765284 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.765180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-token\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.765408 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.765387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5x5t\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-kube-api-access-p5x5t\") pod \"e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") " pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:03.887431 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:03.887337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:04.170604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:04.170572    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"]
Sep 12 01:58:04.172636 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:04.172607    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1df4bc8_7411_4f58_82f4_6c19c0c66381.slice/crio-273213ab09be7e0aac31d28866e274e8933a1d3e4c08187e884d63f0f29ee209 WatchSource:0}: Error finding container 273213ab09be7e0aac31d28866e274e8933a1d3e4c08187e884d63f0f29ee209: Status 404 returned error can't find the container with id 273213ab09be7e0aac31d28866e274e8933a1d3e4c08187e884d63f0f29ee209
Sep 12 01:58:04.175186 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:04.175151    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:04.175293 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:04.175237    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:04.175293 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:04.175270    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:05.080771 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.080725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" event={"ID":"e1df4bc8-7411-4f58-82f4-6c19c0c66381","Type":"ContainerStarted","Data":"8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079"}
Sep 12 01:58:05.081170 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.080793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" event={"ID":"e1df4bc8-7411-4f58-82f4-6c19c0c66381","Type":"ContainerStarted","Data":"273213ab09be7e0aac31d28866e274e8933a1d3e4c08187e884d63f0f29ee209"}
Sep 12 01:58:05.114917 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.114865    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podStartSLOduration=2.114848324 podStartE2EDuration="2.114848324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:05.111869296 +0000 UTC m=+790.638770388" watchObservedRunningTime="2026-09-12 01:58:05.114848324 +0000 UTC m=+790.641749403"
Sep 12 01:58:05.225798 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.225767    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8075-e7490b"]
Sep 12 01:58:05.229985 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.229951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8075-e7490b"
Sep 12 01:58:05.233800 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.233775    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dng96\""
Sep 12 01:58:05.238370 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.238345    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8075-e7490b"]
Sep 12 01:58:05.370912 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.370821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv5d9\" (UniqueName: \"kubernetes.io/projected/a757fcdb-d2df-4d64-b24e-6e7985ca78da-kube-api-access-kv5d9\") pod \"test-curl-8075-e7490b\" (UID: \"a757fcdb-d2df-4d64-b24e-6e7985ca78da\") " pod="opendatahub/test-curl-8075-e7490b"
Sep 12 01:58:05.471832 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.471793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv5d9\" (UniqueName: \"kubernetes.io/projected/a757fcdb-d2df-4d64-b24e-6e7985ca78da-kube-api-access-kv5d9\") pod \"test-curl-8075-e7490b\" (UID: \"a757fcdb-d2df-4d64-b24e-6e7985ca78da\") " pod="opendatahub/test-curl-8075-e7490b"
Sep 12 01:58:05.480729 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.480697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv5d9\" (UniqueName: \"kubernetes.io/projected/a757fcdb-d2df-4d64-b24e-6e7985ca78da-kube-api-access-kv5d9\") pod \"test-curl-8075-e7490b\" (UID: \"a757fcdb-d2df-4d64-b24e-6e7985ca78da\") " pod="opendatahub/test-curl-8075-e7490b"
Sep 12 01:58:05.540777 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.540746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8075-e7490b"
Sep 12 01:58:05.825201 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.825171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8075-e7490b"]
Sep 12 01:58:05.826680 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:05.826653    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda757fcdb_d2df_4d64_b24e_6e7985ca78da.slice/crio-45fa7e900e86899c446f544b34d51a94f3e2b5c82fcf3ce3339ec89be4c7b968 WatchSource:0}: Error finding container 45fa7e900e86899c446f544b34d51a94f3e2b5c82fcf3ce3339ec89be4c7b968: Status 404 returned error can't find the container with id 45fa7e900e86899c446f544b34d51a94f3e2b5c82fcf3ce3339ec89be4c7b968
Sep 12 01:58:05.888506 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.888473    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:05.893972 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:05.893950    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:06.088782 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:06.088694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8075-e7490b" event={"ID":"a757fcdb-d2df-4d64-b24e-6e7985ca78da","Type":"ContainerStarted","Data":"45fa7e900e86899c446f544b34d51a94f3e2b5c82fcf3ce3339ec89be4c7b968"}
Sep 12 01:58:06.089150 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:06.088841    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:06.089826 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:06.089802    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:09.107954 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.107917    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8075-e7490b" event={"ID":"a757fcdb-d2df-4d64-b24e-6e7985ca78da","Type":"ContainerStarted","Data":"bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac"}
Sep 12 01:58:09.125287 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.125238    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8075-e7490b" podStartSLOduration=1.683173117 podStartE2EDuration="4.12522263s" totalImagesPullingTime="2.442049513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:09.1241133 +0000 UTC m=+794.651014380" watchObservedRunningTime="2026-09-12 01:58:09.12522263 +0000 UTC m=+794.652123713"
Sep 12 01:58:09.781423 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.781386    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8075-e7490b"]
Sep 12 01:58:09.788916 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.788888    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8075-e7490b"]
Sep 12 01:58:09.912559 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.912475    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8075-8d7ec2"]
Sep 12 01:58:09.912866 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.912853    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a757fcdb-d2df-4d64-b24e-6e7985ca78da" containerName="test-curl-8075-e7490b"
Sep 12 01:58:09.912916 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.912868    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a757fcdb-d2df-4d64-b24e-6e7985ca78da" containerName="test-curl-8075-e7490b"
Sep 12 01:58:09.912958 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.912927    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a757fcdb-d2df-4d64-b24e-6e7985ca78da" containerName="test-curl-8075-e7490b"
Sep 12 01:58:09.916193 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.916173    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8075-8d7ec2"
Sep 12 01:58:09.925035 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:09.925008    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8075-8d7ec2"]
Sep 12 01:58:10.011491 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.011453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgjr7\" (UniqueName: \"kubernetes.io/projected/e4c264c3-f8f0-4716-a368-36960c287950-kube-api-access-xgjr7\") pod \"test-curl-8075-8d7ec2\" (UID: \"e4c264c3-f8f0-4716-a368-36960c287950\") " pod="opendatahub/test-curl-8075-8d7ec2"
Sep 12 01:58:10.112814 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.112775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgjr7\" (UniqueName: \"kubernetes.io/projected/e4c264c3-f8f0-4716-a368-36960c287950-kube-api-access-xgjr7\") pod \"test-curl-8075-8d7ec2\" (UID: \"e4c264c3-f8f0-4716-a368-36960c287950\") " pod="opendatahub/test-curl-8075-8d7ec2"
Sep 12 01:58:10.114062 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:58:10.114039    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8075-e7490b" container="test-curl-8075-e7490b"
Sep 12 01:58:10.122037 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.122010    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgjr7\" (UniqueName: \"kubernetes.io/projected/e4c264c3-f8f0-4716-a368-36960c287950-kube-api-access-xgjr7\") pod \"test-curl-8075-8d7ec2\" (UID: \"e4c264c3-f8f0-4716-a368-36960c287950\") " pod="opendatahub/test-curl-8075-8d7ec2"
Sep 12 01:58:10.226490 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.226399    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8075-8d7ec2"
Sep 12 01:58:10.290780 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.289388    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"]
Sep 12 01:58:10.294341 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.294316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:10.301285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.298500    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-3fc028-dockercfg-hdnds\""
Sep 12 01:58:10.301285 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.300130    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-3fc028\""
Sep 12 01:58:10.304405 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.304366    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"]
Sep 12 01:58:10.415631 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.415589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brbns\" (UniqueName: \"kubernetes.io/projected/2e36106c-a86c-47e3-9ad5-3d179946557b-kube-api-access-brbns\") pod \"payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s\" (UID: \"2e36106c-a86c-47e3-9ad5-3d179946557b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:10.415631 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.415636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e36106c-a86c-47e3-9ad5-3d179946557b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s\" (UID: \"2e36106c-a86c-47e3-9ad5-3d179946557b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:10.516481 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.516449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brbns\" (UniqueName: \"kubernetes.io/projected/2e36106c-a86c-47e3-9ad5-3d179946557b-kube-api-access-brbns\") pod \"payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s\" (UID: \"2e36106c-a86c-47e3-9ad5-3d179946557b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:10.516481 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.516485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e36106c-a86c-47e3-9ad5-3d179946557b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s\" (UID: \"2e36106c-a86c-47e3-9ad5-3d179946557b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:10.517162 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.517141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e36106c-a86c-47e3-9ad5-3d179946557b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s\" (UID: \"2e36106c-a86c-47e3-9ad5-3d179946557b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:10.531901 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:10.531869    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4c264c3_f8f0_4716_a368_36960c287950.slice/crio-28ec3e82c17d101829755124328855e6c5fd155acadffd6430627d0015026974 WatchSource:0}: Error finding container 28ec3e82c17d101829755124328855e6c5fd155acadffd6430627d0015026974: Status 404 returned error can't find the container with id 28ec3e82c17d101829755124328855e6c5fd155acadffd6430627d0015026974
Sep 12 01:58:10.532614 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.532589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8075-8d7ec2"]
Sep 12 01:58:10.535784 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.535761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brbns\" (UniqueName: \"kubernetes.io/projected/2e36106c-a86c-47e3-9ad5-3d179946557b-kube-api-access-brbns\") pod \"payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s\" (UID: \"2e36106c-a86c-47e3-9ad5-3d179946557b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:10.609756 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.609720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:10.896081 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:10.893319    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"]
Sep 12 01:58:10.896902 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:10.896859    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e36106c_a86c_47e3_9ad5_3d179946557b.slice/crio-0023473077f2f46f7ae2bc56cbb28fbc7bd02bdfa9ceaf0891917ea31d7bbb9d WatchSource:0}: Error finding container 0023473077f2f46f7ae2bc56cbb28fbc7bd02bdfa9ceaf0891917ea31d7bbb9d: Status 404 returned error can't find the container with id 0023473077f2f46f7ae2bc56cbb28fbc7bd02bdfa9ceaf0891917ea31d7bbb9d
Sep 12 01:58:11.120751 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.120713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s" event={"ID":"2e36106c-a86c-47e3-9ad5-3d179946557b","Type":"ContainerStarted","Data":"0023473077f2f46f7ae2bc56cbb28fbc7bd02bdfa9ceaf0891917ea31d7bbb9d"}
Sep 12 01:58:11.122410 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.122375    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8075-8d7ec2" event={"ID":"e4c264c3-f8f0-4716-a368-36960c287950","Type":"ContainerStarted","Data":"caafe1391609ff97b52d61cf270903031c1eb2f00280cf39ce2debbb1a71bd81"}
Sep 12 01:58:11.122410 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.122418    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8075-8d7ec2" event={"ID":"e4c264c3-f8f0-4716-a368-36960c287950","Type":"ContainerStarted","Data":"28ec3e82c17d101829755124328855e6c5fd155acadffd6430627d0015026974"}
Sep 12 01:58:11.122623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.122481    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8075-e7490b" podUID="a757fcdb-d2df-4d64-b24e-6e7985ca78da" containerName="test-curl-8075-e7490b" containerID="cri-o://bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac" gracePeriod=2
Sep 12 01:58:11.138767 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.138724    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8075-8d7ec2" podStartSLOduration=2.138708412 podStartE2EDuration="2.138708412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:11.137945113 +0000 UTC m=+796.664846191" watchObservedRunningTime="2026-09-12 01:58:11.138708412 +0000 UTC m=+796.665609520"
Sep 12 01:58:11.616859 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.616771    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"]
Sep 12 01:58:11.620466 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.620444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:11.623505 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.623480    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-98f68e\""
Sep 12 01:58:11.623623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.623568    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-98f68e-dockercfg-k9ddg\""
Sep 12 01:58:11.635566 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.635544    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"]
Sep 12 01:58:11.726732 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.726693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91238106-7be6-44c5-881f-fe690dbc7c69-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx\" (UID: \"91238106-7be6-44c5-881f-fe690dbc7c69\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:11.726932 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.726802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc787\" (UniqueName: \"kubernetes.io/projected/91238106-7be6-44c5-881f-fe690dbc7c69-kube-api-access-wc787\") pod \"payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx\" (UID: \"91238106-7be6-44c5-881f-fe690dbc7c69\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:11.828026 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.827984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91238106-7be6-44c5-881f-fe690dbc7c69-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx\" (UID: \"91238106-7be6-44c5-881f-fe690dbc7c69\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:11.828273 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.828080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc787\" (UniqueName: \"kubernetes.io/projected/91238106-7be6-44c5-881f-fe690dbc7c69-kube-api-access-wc787\") pod \"payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx\" (UID: \"91238106-7be6-44c5-881f-fe690dbc7c69\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:11.828902 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.828880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91238106-7be6-44c5-881f-fe690dbc7c69-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx\" (UID: \"91238106-7be6-44c5-881f-fe690dbc7c69\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:11.839435 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.839411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc787\" (UniqueName: \"kubernetes.io/projected/91238106-7be6-44c5-881f-fe690dbc7c69-kube-api-access-wc787\") pod \"payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx\" (UID: \"91238106-7be6-44c5-881f-fe690dbc7c69\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:11.911177 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.911089    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8075-8d7ec2"]
Sep 12 01:58:11.916576 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.916395    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8075-8d7ec2"]
Sep 12 01:58:11.934981 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:11.934949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:12.130253 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:58:12.130137    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8075-8d7ec2" container="test-curl-8075-8d7ec2"
Sep 12 01:58:12.319776 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.319751    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"]
Sep 12 01:58:12.321466 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:12.321436    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91238106_7be6_44c5_881f_fe690dbc7c69.slice/crio-1fb931e6016d00d8fd9064504da8c58b2987a4ed94499cf3102ed001bf2c4001 WatchSource:0}: Error finding container 1fb931e6016d00d8fd9064504da8c58b2987a4ed94499cf3102ed001bf2c4001: Status 404 returned error can't find the container with id 1fb931e6016d00d8fd9064504da8c58b2987a4ed94499cf3102ed001bf2c4001
Sep 12 01:58:12.778584 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.778550    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"]
Sep 12 01:58:12.778941 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.778926    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4c264c3-f8f0-4716-a368-36960c287950" containerName="test-curl-8075-8d7ec2"
Sep 12 01:58:12.778941 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.778942    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4c264c3-f8f0-4716-a368-36960c287950" containerName="test-curl-8075-8d7ec2"
Sep 12 01:58:12.779031 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.779009    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4c264c3-f8f0-4716-a368-36960c287950" containerName="test-curl-8075-8d7ec2"
Sep 12 01:58:12.782074 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.782056    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:12.784862 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.784841    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-9ef13e-openshift-default-dockercfg-9h6jp\""
Sep 12 01:58:12.800573 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.800547    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"]
Sep 12 01:58:12.938445 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.938408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8252df-28cf-4c63-be91-91bab798d297-istiod-ca-cert\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:12.938445 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.938451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-credential-socket\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:12.938678 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.938473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-certs\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:12.938678 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.938543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl5p7\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-kube-api-access-sl5p7\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:12.938678 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.938566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-socket\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:12.938678 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.938600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-envoy\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:12.938678 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.938677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-istio-token\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:12.938865 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.938701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8252df-28cf-4c63-be91-91bab798d297-istio-podinfo\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:12.938865 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:12.938730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-data\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.018649 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.018614    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"]
Sep 12 01:58:13.022896 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.022867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.026095 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.026066    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-57c4f4a4-openshift-default-dockercfg-pcq42\""
Sep 12 01:58:13.034339 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.034309    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"]
Sep 12 01:58:13.040049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-istio-token\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040175 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8252df-28cf-4c63-be91-91bab798d297-istio-podinfo\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040175 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-data\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040175 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8252df-28cf-4c63-be91-91bab798d297-istiod-ca-cert\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040175 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-credential-socket\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040413 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-certs\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040413 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl5p7\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-kube-api-access-sl5p7\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040413 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-socket\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040413 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-envoy\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040610 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-data\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040736 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-certs\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.040879 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.040801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-socket\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.041191 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.041170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-credential-socket\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.041344 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.041262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8252df-28cf-4c63-be91-91bab798d297-istiod-ca-cert\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.043645 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.043620    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-envoy\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.043745 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.043627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8252df-28cf-4c63-be91-91bab798d297-istio-podinfo\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.052489 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.052464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-istio-token\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.053802 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.053781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl5p7\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-kube-api-access-sl5p7\") pod \"e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") " pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.092719 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.092687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:13.139655 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.139619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx" event={"ID":"91238106-7be6-44c5-881f-fe690dbc7c69","Type":"ContainerStarted","Data":"2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978"}
Sep 12 01:58:13.140026 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.139687    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx" event={"ID":"91238106-7be6-44c5-881f-fe690dbc7c69","Type":"ContainerStarted","Data":"1fb931e6016d00d8fd9064504da8c58b2987a4ed94499cf3102ed001bf2c4001"}
Sep 12 01:58:13.140026 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.139898    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:13.140951 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.140930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-envoy\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.141055 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.140981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-data\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.141055 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.141008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-istio-token\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.141169 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.141068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-socket\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.141169 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.141094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-credential-socket\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.141169 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.141146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07996748-3c28-4ad3-8865-ed7073d88a42-istio-podinfo\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.141399 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.141240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07996748-3c28-4ad3-8865-ed7073d88a42-istiod-ca-cert\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.141399 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.141275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-certs\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.141399 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.141302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96nj9\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-kube-api-access-96nj9\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.142051 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.142024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s" event={"ID":"2e36106c-a86c-47e3-9ad5-3d179946557b","Type":"ContainerStarted","Data":"399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38"}
Sep 12 01:58:13.142137 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.142092    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8075-8d7ec2" podUID="e4c264c3-f8f0-4716-a368-36960c287950" containerName="test-curl-8075-8d7ec2" containerID="cri-o://caafe1391609ff97b52d61cf270903031c1eb2f00280cf39ce2debbb1a71bd81" gracePeriod=2
Sep 12 01:58:13.142197 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.142173    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:13.163695 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.163645    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx" podStartSLOduration=2.163625124 podStartE2EDuration="2.163625124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:13.160000406 +0000 UTC m=+798.686901487" watchObservedRunningTime="2026-09-12 01:58:13.163625124 +0000 UTC m=+798.690526205"
Sep 12 01:58:13.178318 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.178265    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s" podStartSLOduration=1.9758503539999999 podStartE2EDuration="3.178246196s" totalImagesPullingTime="1.202395842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:13.176305302 +0000 UTC m=+798.703206382" watchObservedRunningTime="2026-09-12 01:58:13.178246196 +0000 UTC m=+798.705147276"
Sep 12 01:58:13.243380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.242426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-envoy\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.242486    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-data\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.242512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-istio-token\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.242565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-socket\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.242591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-credential-socket\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.242637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07996748-3c28-4ad3-8865-ed7073d88a42-istio-podinfo\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.242682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07996748-3c28-4ad3-8865-ed7073d88a42-istiod-ca-cert\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.242713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-certs\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243380 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.242740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96nj9\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-kube-api-access-96nj9\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243999 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.243783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-certs\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.243999 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.243847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-socket\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.244654 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.244442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-data\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.244811 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.244662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-credential-socket\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.244981 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.244953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07996748-3c28-4ad3-8865-ed7073d88a42-istiod-ca-cert\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.245704 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.245658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-envoy\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.246924 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.246902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07996748-3c28-4ad3-8865-ed7073d88a42-istio-podinfo\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.254451 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.254368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96nj9\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-kube-api-access-96nj9\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.256618 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.256596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-istio-token\") pod \"e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") " pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.319932 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.319909    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8075-e7490b"
Sep 12 01:58:13.335782 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.335757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:13.377521 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.377482    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"]
Sep 12 01:58:13.385314 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.385273    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:13.385439 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.385345    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:13.385439 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.385382    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:13.445155 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.445006    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv5d9\" (UniqueName: \"kubernetes.io/projected/a757fcdb-d2df-4d64-b24e-6e7985ca78da-kube-api-access-kv5d9\") pod \"a757fcdb-d2df-4d64-b24e-6e7985ca78da\" (UID: \"a757fcdb-d2df-4d64-b24e-6e7985ca78da\") "
Sep 12 01:58:13.447346 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.447306    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a757fcdb-d2df-4d64-b24e-6e7985ca78da-kube-api-access-kv5d9" (OuterVolumeSpecName: "kube-api-access-kv5d9") pod "a757fcdb-d2df-4d64-b24e-6e7985ca78da" (UID: "a757fcdb-d2df-4d64-b24e-6e7985ca78da"). InnerVolumeSpecName "kube-api-access-kv5d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:58:13.546634 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.546596    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv5d9\" (UniqueName: \"kubernetes.io/projected/a757fcdb-d2df-4d64-b24e-6e7985ca78da-kube-api-access-kv5d9\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:13.626808 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.626773    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"]
Sep 12 01:58:13.627835 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:13.627803    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07996748_3c28_4ad3_8865_ed7073d88a42.slice/crio-6dcd8031477f44423a66b8dcd0e09d57d270655ba4168e9087137bf2a529121b WatchSource:0}: Error finding container 6dcd8031477f44423a66b8dcd0e09d57d270655ba4168e9087137bf2a529121b: Status 404 returned error can't find the container with id 6dcd8031477f44423a66b8dcd0e09d57d270655ba4168e9087137bf2a529121b
Sep 12 01:58:13.630365 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.630331    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:13.630480 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.630410    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:13.630480 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:13.630457    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:14.153330 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.153299    2580 generic.go:356] "Generic (PLEG): container finished" podID="a757fcdb-d2df-4d64-b24e-6e7985ca78da" containerID="bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac" exitCode=137
Sep 12 01:58:14.153785 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.153351    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8075-e7490b"
Sep 12 01:58:14.153785 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.153431    2580 scope.go:117] "RemoveContainer" containerID="bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac"
Sep 12 01:58:14.155652 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.155626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" event={"ID":"8f8252df-28cf-4c63-be91-91bab798d297","Type":"ContainerStarted","Data":"6857e9ec9d826bd3009835995e89b1d70ca85709990a70d240d93806fbd63eeb"}
Sep 12 01:58:14.155774 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.155701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" event={"ID":"8f8252df-28cf-4c63-be91-91bab798d297","Type":"ContainerStarted","Data":"cc98e32f614b5723c3f388d6d771e0a4ea13e9d5f80b95345b85944895279192"}
Sep 12 01:58:14.157678 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.157655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" event={"ID":"07996748-3c28-4ad3-8865-ed7073d88a42","Type":"ContainerStarted","Data":"3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306"}
Sep 12 01:58:14.157783 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.157712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" event={"ID":"07996748-3c28-4ad3-8865-ed7073d88a42","Type":"ContainerStarted","Data":"6dcd8031477f44423a66b8dcd0e09d57d270655ba4168e9087137bf2a529121b"}
Sep 12 01:58:14.165250 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.165232    2580 scope.go:117] "RemoveContainer" containerID="bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac"
Sep 12 01:58:14.165571 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:58:14.165548    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac\": container with ID starting with bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac not found: ID does not exist" containerID="bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac"
Sep 12 01:58:14.165616 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.165583    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac"} err="failed to get container status \"bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac\": rpc error: code = NotFound desc = could not find container \"bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac\": container with ID starting with bc2a7c6f90e8cf1ea9995dfd4704c44f30b3f8183696ef1dba54f354d5c59cac not found: ID does not exist"
Sep 12 01:58:14.183916 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.183856    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" podStartSLOduration=2.183835868 podStartE2EDuration="2.183835868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:14.180044521 +0000 UTC m=+799.706945602" watchObservedRunningTime="2026-09-12 01:58:14.183835868 +0000 UTC m=+799.710736951"
Sep 12 01:58:14.202429 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.202189    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" podStartSLOduration=2.202168373 podStartE2EDuration="2.202168373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:14.200764746 +0000 UTC m=+799.727665826" watchObservedRunningTime="2026-09-12 01:58:14.202168373 +0000 UTC m=+799.729069457"
Sep 12 01:58:14.207562 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:58:14.207526    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8075-e7490b\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="a757fcdb-d2df-4d64-b24e-6e7985ca78da" pod="opendatahub/test-curl-8075-e7490b"
Sep 12 01:58:14.336147 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.336111    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:14.337958 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.337926    2580 patch_prober.go:28] interesting pod/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 12 01:58:14.338117 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.337989    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 12 01:58:14.979443 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:58:14.979389    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8075-e7490b\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="a757fcdb-d2df-4d64-b24e-6e7985ca78da" pod="opendatahub/test-curl-8075-e7490b"
Sep 12 01:58:14.979705 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:14.979680    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a757fcdb-d2df-4d64-b24e-6e7985ca78da" path="/var/lib/kubelet/pods/a757fcdb-d2df-4d64-b24e-6e7985ca78da/volumes"
Sep 12 01:58:15.093715 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.093674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:15.099368 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.099340    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:15.168021 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.167984    2580 generic.go:356] "Generic (PLEG): container finished" podID="e4c264c3-f8f0-4716-a368-36960c287950" containerID="caafe1391609ff97b52d61cf270903031c1eb2f00280cf39ce2debbb1a71bd81" exitCode=137
Sep 12 01:58:15.170321 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.170016    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:15.170985 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.170966    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:15.314841 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.314815    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8075-8d7ec2"
Sep 12 01:58:15.341065 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.341036    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:15.465505 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.465458    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgjr7\" (UniqueName: \"kubernetes.io/projected/e4c264c3-f8f0-4716-a368-36960c287950-kube-api-access-xgjr7\") pod \"e4c264c3-f8f0-4716-a368-36960c287950\" (UID: \"e4c264c3-f8f0-4716-a368-36960c287950\") "
Sep 12 01:58:15.467882 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.467846    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4c264c3-f8f0-4716-a368-36960c287950-kube-api-access-xgjr7" (OuterVolumeSpecName: "kube-api-access-xgjr7") pod "e4c264c3-f8f0-4716-a368-36960c287950" (UID: "e4c264c3-f8f0-4716-a368-36960c287950"). InnerVolumeSpecName "kube-api-access-xgjr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:58:15.567112 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:15.567026    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgjr7\" (UniqueName: \"kubernetes.io/projected/e4c264c3-f8f0-4716-a368-36960c287950-kube-api-access-xgjr7\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:16.176822 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:16.176795    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8075-8d7ec2"
Sep 12 01:58:16.177332 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:16.176846    2580 scope.go:117] "RemoveContainer" containerID="caafe1391609ff97b52d61cf270903031c1eb2f00280cf39ce2debbb1a71bd81"
Sep 12 01:58:16.177430 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:16.177406    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:16.178631 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:16.178611    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:16.201780 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:58:16.201749    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8075-8d7ec2\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="e4c264c3-f8f0-4716-a368-36960c287950" pod="opendatahub/test-curl-8075-8d7ec2"
Sep 12 01:58:16.979853 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:16.979815    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4c264c3-f8f0-4716-a368-36960c287950" path="/var/lib/kubelet/pods/e4c264c3-f8f0-4716-a368-36960c287950/volumes"
Sep 12 01:58:20.627173 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.627095    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"]
Sep 12 01:58:20.630799 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.630778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:20.633566 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.633543    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-9ef13e\""
Sep 12 01:58:20.633676 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.633605    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-9ef13e-dockercfg-vk9q9\""
Sep 12 01:58:20.641427 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.641401    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"]
Sep 12 01:58:20.819521 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.819479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7be6a912-f702-406c-ab94-bc038b3c5513-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z\" (UID: \"7be6a912-f702-406c-ab94-bc038b3c5513\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:20.819711 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.819533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsqqh\" (UniqueName: \"kubernetes.io/projected/7be6a912-f702-406c-ab94-bc038b3c5513-kube-api-access-lsqqh\") pod \"payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z\" (UID: \"7be6a912-f702-406c-ab94-bc038b3c5513\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:20.921070 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.920964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7be6a912-f702-406c-ab94-bc038b3c5513-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z\" (UID: \"7be6a912-f702-406c-ab94-bc038b3c5513\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:20.921070 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.921016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsqqh\" (UniqueName: \"kubernetes.io/projected/7be6a912-f702-406c-ab94-bc038b3c5513-kube-api-access-lsqqh\") pod \"payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z\" (UID: \"7be6a912-f702-406c-ab94-bc038b3c5513\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:20.921743 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.921722    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7be6a912-f702-406c-ab94-bc038b3c5513-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z\" (UID: \"7be6a912-f702-406c-ab94-bc038b3c5513\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:20.931299 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.931272    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsqqh\" (UniqueName: \"kubernetes.io/projected/7be6a912-f702-406c-ab94-bc038b3c5513-kube-api-access-lsqqh\") pod \"payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z\" (UID: \"7be6a912-f702-406c-ab94-bc038b3c5513\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:20.942946 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:20.942919    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:21.245853 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:21.245825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"]
Sep 12 01:58:21.246989 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:21.246952    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be6a912_f702_406c_ab94_bc038b3c5513.slice/crio-573f53c6cb1e964ecc3cd0b97f2228acd9e6550df2c89ce917a4141f7bdfd5a6 WatchSource:0}: Error finding container 573f53c6cb1e964ecc3cd0b97f2228acd9e6550df2c89ce917a4141f7bdfd5a6: Status 404 returned error can't find the container with id 573f53c6cb1e964ecc3cd0b97f2228acd9e6550df2c89ce917a4141f7bdfd5a6
Sep 12 01:58:21.809940 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:21.809900    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"]
Sep 12 01:58:21.813640 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:21.813618    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:21.816347 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:21.816323    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-57c4f4a4-dockercfg-vthgv\""
Sep 12 01:58:21.816452 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:21.816355    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-57c4f4a4\""
Sep 12 01:58:21.825162 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:21.825126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"]
Sep 12 01:58:21.930840 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:21.930803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-kube-api-access-sm94h\") pod \"payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk\" (UID: \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:21.930995 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:21.930858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk\" (UID: \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:22.032018 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.031980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk\" (UID: \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:22.032242 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.032134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-kube-api-access-sm94h\") pod \"payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk\" (UID: \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:22.032819 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.032794    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk\" (UID: \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:22.052938 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.052903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-kube-api-access-sm94h\") pod \"payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk\" (UID: \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:22.127805 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.127722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:22.230828 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.230795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z" event={"ID":"7be6a912-f702-406c-ab94-bc038b3c5513","Type":"ContainerStarted","Data":"a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621"}
Sep 12 01:58:22.230968 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.230853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z" event={"ID":"7be6a912-f702-406c-ab94-bc038b3c5513","Type":"ContainerStarted","Data":"573f53c6cb1e964ecc3cd0b97f2228acd9e6550df2c89ce917a4141f7bdfd5a6"}
Sep 12 01:58:22.231028 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.231005    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:22.442964 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.442911    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z" podStartSLOduration=2.442893674 podStartE2EDuration="2.442893674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:22.276524572 +0000 UTC m=+807.803425653" watchObservedRunningTime="2026-09-12 01:58:22.442893674 +0000 UTC m=+807.969794761"
Sep 12 01:58:22.444233 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:22.444193    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"]
Sep 12 01:58:22.444976 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:22.444951    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8e77ad_54e2_4d0d_aea2_24b8f41787dd.slice/crio-b51d8a659b6ceae3acb748d8058d3f862d07d6e2895f1fab83fe851ffbe64879 WatchSource:0}: Error finding container b51d8a659b6ceae3acb748d8058d3f862d07d6e2895f1fab83fe851ffbe64879: Status 404 returned error can't find the container with id b51d8a659b6ceae3acb748d8058d3f862d07d6e2895f1fab83fe851ffbe64879
Sep 12 01:58:23.238538 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:23.238502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk" event={"ID":"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd","Type":"ContainerStarted","Data":"0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab"}
Sep 12 01:58:23.238946 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:23.238639    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:23.238946 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:23.238660    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk" event={"ID":"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd","Type":"ContainerStarted","Data":"b51d8a659b6ceae3acb748d8058d3f862d07d6e2895f1fab83fe851ffbe64879"}
Sep 12 01:58:23.256759 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:23.256708    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk" podStartSLOduration=2.256691959 podStartE2EDuration="2.256691959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:23.256041414 +0000 UTC m=+808.782942493" watchObservedRunningTime="2026-09-12 01:58:23.256691959 +0000 UTC m=+808.783593038"
Sep 12 01:58:24.159768 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:24.159720    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 01:58:24.159910 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:24.159831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 01:58:27.406879 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.406842    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"]
Sep 12 01:58:27.417946 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.417917    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.421240 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.421195    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7d6e9f-openshift-default-dockercfg-6m2xx\""
Sep 12 01:58:27.440341 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.439405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"]
Sep 12 01:58:27.488784 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.488741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-token\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.489103 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.489077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-credential-socket\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.489321 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.489302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b17d3b35-603b-465c-aed7-e0ee82447dda-istiod-ca-cert\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.489481 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.489464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd6cw\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-kube-api-access-gd6cw\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.489671 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.489653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-certs\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.489965 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.489943    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-podinfo\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.490138 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.490120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-data\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.490328 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.490313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-envoy\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.490486 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.490469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-socket\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.593739 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.593699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-token\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.593739 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.593757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-credential-socket\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.594016 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.593793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b17d3b35-603b-465c-aed7-e0ee82447dda-istiod-ca-cert\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.594016 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.593822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd6cw\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-kube-api-access-gd6cw\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.594016 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.593863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-certs\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.594016 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.593948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-podinfo\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.594016 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.593982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-data\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.594016 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.594013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-envoy\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.594360 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.594041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-socket\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.595102 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.594520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-socket\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.595102 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.594778    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-certs\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.595102 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.595020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-data\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.595388 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.595283    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-credential-socket\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.595960 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.595908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b17d3b35-603b-465c-aed7-e0ee82447dda-istiod-ca-cert\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.611118 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.611081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-podinfo\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.612427 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.612392    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd6cw\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-kube-api-access-gd6cw\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.612703 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.612680    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-envoy\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.613104 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.613078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-token\") pod \"e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") " pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:27.744916 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:27.744874    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:28.121295 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:28.118956    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"]
Sep 12 01:58:28.129016 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:28.128889    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:28.129016 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:28.128989    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:28.129105 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:28.129044    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:28.285232 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:28.285035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" event={"ID":"b17d3b35-603b-465c-aed7-e0ee82447dda","Type":"ContainerStarted","Data":"d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5"}
Sep 12 01:58:28.285232 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:28.285192    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" event={"ID":"b17d3b35-603b-465c-aed7-e0ee82447dda","Type":"ContainerStarted","Data":"5435d35b6e587aed21f82c22e28d55964a5ac8a397b8f54b92a1e0919a337389"}
Sep 12 01:58:28.310026 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:28.309805    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" podStartSLOduration=1.309782778 podStartE2EDuration="1.309782778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:28.30680371 +0000 UTC m=+813.833704792" watchObservedRunningTime="2026-09-12 01:58:28.309782778 +0000 UTC m=+813.836683863"
Sep 12 01:58:28.745401 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:28.745362    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:29.750766 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:29.750730    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:30.314237 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:30.314105    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:30.326266 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:30.324505    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:31.325414 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:31.325248    2580 generic.go:356] "Generic (PLEG): container finished" podID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerID="8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079" exitCode=0
Sep 12 01:58:31.325972 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:31.325947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" event={"ID":"e1df4bc8-7411-4f58-82f4-6c19c0c66381","Type":"ContainerDied","Data":"8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079"}
Sep 12 01:58:31.326585 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:31.326568    2580 scope.go:117] "RemoveContainer" containerID="8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079"
Sep 12 01:58:31.327811 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:31.327781    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:31.327862 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:31.327849    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:31.327897 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:31.327881    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:32.335198 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:32.335157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" event={"ID":"e1df4bc8-7411-4f58-82f4-6c19c0c66381","Type":"ContainerStarted","Data":"2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2"}
Sep 12 01:58:32.887785 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:32.887746    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:32.889252 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:32.889223    2580 patch_prober.go:28] interesting pod/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 12 01:58:32.889397 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:32.889308    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 12 01:58:33.241161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:33.241069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 01:58:33.888422 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:33.888376    2580 patch_prober.go:28] interesting pod/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 12 01:58:33.889022 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:33.888985    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 12 01:58:34.249172 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:34.248762    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:34.888723 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:34.888678    2580 patch_prober.go:28] interesting pod/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 12 01:58:34.889356 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:34.889322    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 12 01:58:35.889061 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:35.888590    2580 patch_prober.go:28] interesting pod/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 12 01:58:35.889061 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:35.888662    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 12 01:58:36.089325 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.089289    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:36.103633 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.103596    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"]
Sep 12 01:58:36.106327 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.106304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"
Sep 12 01:58:36.108867 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.108845    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 01:58:36.108986 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.108846    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 01:58:36.114319 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.114290    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"]
Sep 12 01:58:36.212332 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.212201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e290a594-43d4-4b2f-9a18-01ddc4d24600-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg\" (UID: \"e290a594-43d4-4b2f-9a18-01ddc4d24600\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"
Sep 12 01:58:36.314013 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.313965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e290a594-43d4-4b2f-9a18-01ddc4d24600-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg\" (UID: \"e290a594-43d4-4b2f-9a18-01ddc4d24600\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"
Sep 12 01:58:36.314615 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.314593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e290a594-43d4-4b2f-9a18-01ddc4d24600-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg\" (UID: \"e290a594-43d4-4b2f-9a18-01ddc4d24600\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"
Sep 12 01:58:36.370806 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.369449    2580 generic.go:356] "Generic (PLEG): container finished" podID="07996748-3c28-4ad3-8865-ed7073d88a42" containerID="3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306" exitCode=0
Sep 12 01:58:36.370806 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.369587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" event={"ID":"07996748-3c28-4ad3-8865-ed7073d88a42","Type":"ContainerDied","Data":"3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306"}
Sep 12 01:58:36.370806 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.370067    2580 scope.go:117] "RemoveContainer" containerID="3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306"
Sep 12 01:58:36.379238 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.372754    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:36.379238 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.372853    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:36.379238 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.372905    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:36.420653 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.420623    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"
Sep 12 01:58:36.741717 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.741687    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"]
Sep 12 01:58:36.743747 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:36.743689    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode290a594_43d4_4b2f_9a18_01ddc4d24600.slice/crio-f4f442a76ceee8f1ba16310a98a2b2de2cc6cb6d4ec8e81ba37eb2c0a17162c9 WatchSource:0}: Error finding container f4f442a76ceee8f1ba16310a98a2b2de2cc6cb6d4ec8e81ba37eb2c0a17162c9: Status 404 returned error can't find the container with id f4f442a76ceee8f1ba16310a98a2b2de2cc6cb6d4ec8e81ba37eb2c0a17162c9
Sep 12 01:58:36.888733 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.888690    2580 patch_prober.go:28] interesting pod/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 12 01:58:36.888952 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:36.888766    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 12 01:58:37.386653 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.386556    2580 generic.go:356] "Generic (PLEG): container finished" podID="8f8252df-28cf-4c63-be91-91bab798d297" containerID="6857e9ec9d826bd3009835995e89b1d70ca85709990a70d240d93806fbd63eeb" exitCode=0
Sep 12 01:58:37.387253 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.386638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" event={"ID":"8f8252df-28cf-4c63-be91-91bab798d297","Type":"ContainerDied","Data":"6857e9ec9d826bd3009835995e89b1d70ca85709990a70d240d93806fbd63eeb"}
Sep 12 01:58:37.388355 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.388334    2580 scope.go:117] "RemoveContainer" containerID="6857e9ec9d826bd3009835995e89b1d70ca85709990a70d240d93806fbd63eeb"
Sep 12 01:58:37.390093 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.390058    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:37.390525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.390500    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:37.390688 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.390669    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:37.391545 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.391470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg" event={"ID":"e290a594-43d4-4b2f-9a18-01ddc4d24600","Type":"ContainerStarted","Data":"f4f442a76ceee8f1ba16310a98a2b2de2cc6cb6d4ec8e81ba37eb2c0a17162c9"}
Sep 12 01:58:37.407229 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.406487    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" event={"ID":"07996748-3c28-4ad3-8865-ed7073d88a42","Type":"ContainerStarted","Data":"a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015"}
Sep 12 01:58:37.890127 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.889643    2580 patch_prober.go:28] interesting pod/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 12 01:58:37.890127 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:37.889708    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 12 01:58:38.336292 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:38.336253    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:38.336597 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:38.336560    2580 patch_prober.go:28] interesting pod/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 12 01:58:38.336742 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:38.336624    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 12 01:58:38.429484 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:38.428617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" event={"ID":"8f8252df-28cf-4c63-be91-91bab798d297","Type":"ContainerStarted","Data":"5435ac5b0cf947b954ffb632311d77daebe597d845901b263c65e4e497d99332"}
Sep 12 01:58:38.887889 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:38.887840    2580 patch_prober.go:28] interesting pod/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 12 01:58:38.888106 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:38.887919    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 12 01:58:39.095060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:39.094629    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:39.095060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:39.094756    2580 patch_prober.go:28] interesting pod/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 12 01:58:39.095060 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:39.094799    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 12 01:58:39.337101 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:39.336648    2580 patch_prober.go:28] interesting pod/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 12 01:58:39.337101 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:39.336717    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 12 01:58:39.893412 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:39.893367    2580 patch_prober.go:28] interesting pod/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 12 01:58:39.893920 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:39.893442    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 12 01:58:40.104238 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:40.098674    2580 patch_prober.go:28] interesting pod/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 12 01:58:40.104238 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:40.098753    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 12 01:58:40.337122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:40.337077    2580 patch_prober.go:28] interesting pod/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 12 01:58:40.337424 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:40.337144    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 12 01:58:40.909019 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:40.907523    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:40.916232 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:40.916132    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 01:58:41.100239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:41.096845    2580 patch_prober.go:28] interesting pod/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 12 01:58:41.100239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:41.096930    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 12 01:58:41.336706 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:41.336655    2580 patch_prober.go:28] interesting pod/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 12 01:58:41.336908 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:41.336735    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 12 01:58:42.093411 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:42.093367    2580 patch_prober.go:28] interesting pod/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 12 01:58:42.093934 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:42.093437    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 12 01:58:42.336363 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:42.336320    2580 patch_prober.go:28] interesting pod/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 12 01:58:42.336564 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:42.336386    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 12 01:58:42.457313 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:42.457205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg" event={"ID":"e290a594-43d4-4b2f-9a18-01ddc4d24600","Type":"ContainerStarted","Data":"48b9a3f8cd1194c5be20c34fb220e76462d5dea6423507da5161807fc7db342c"}
Sep 12 01:58:42.473588 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:42.473482    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg" podStartSLOduration=1.695483208 podStartE2EDuration="6.473463059s" totalImagesPullingTime="4.777979851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:42.472827019 +0000 UTC m=+827.999728100" watchObservedRunningTime="2026-09-12 01:58:42.473463059 +0000 UTC m=+828.000364143"
Sep 12 01:58:43.093159 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:43.093110    2580 patch_prober.go:28] interesting pod/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 12 01:58:43.093419 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:43.093183    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 12 01:58:43.340045 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:43.340011    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:43.340462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:43.340305    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:43.341026 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:43.341008    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:43.464824 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:43.464736    2580 generic.go:356] "Generic (PLEG): container finished" podID="e290a594-43d4-4b2f-9a18-01ddc4d24600" containerID="48b9a3f8cd1194c5be20c34fb220e76462d5dea6423507da5161807fc7db342c" exitCode=0
Sep 12 01:58:43.464824 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:43.464794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg" event={"ID":"e290a594-43d4-4b2f-9a18-01ddc4d24600","Type":"ContainerDied","Data":"48b9a3f8cd1194c5be20c34fb220e76462d5dea6423507da5161807fc7db342c"}
Sep 12 01:58:44.093539 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:44.093484    2580 patch_prober.go:28] interesting pod/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 12 01:58:44.093766 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:44.093564    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 12 01:58:44.608474 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:44.608449    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"
Sep 12 01:58:44.725070 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:44.724983    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e290a594-43d4-4b2f-9a18-01ddc4d24600-maas-api-service-ca\") pod \"e290a594-43d4-4b2f-9a18-01ddc4d24600\" (UID: \"e290a594-43d4-4b2f-9a18-01ddc4d24600\") "
Sep 12 01:58:44.725429 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:44.725409    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e290a594-43d4-4b2f-9a18-01ddc4d24600-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e290a594-43d4-4b2f-9a18-01ddc4d24600" (UID: "e290a594-43d4-4b2f-9a18-01ddc4d24600"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:58:44.825902 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:44.825860    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e290a594-43d4-4b2f-9a18-01ddc4d24600-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:45.098065 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:45.098032    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:45.098360 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:45.098339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:45.099225 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:45.099189    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 01:58:45.481604 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:45.481572    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"
Sep 12 01:58:45.481797 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:45.481616    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg" event={"ID":"e290a594-43d4-4b2f-9a18-01ddc4d24600","Type":"ContainerDied","Data":"f4f442a76ceee8f1ba16310a98a2b2de2cc6cb6d4ec8e81ba37eb2c0a17162c9"}
Sep 12 01:58:45.481797 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:45.481674    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4f442a76ceee8f1ba16310a98a2b2de2cc6cb6d4ec8e81ba37eb2c0a17162c9"
Sep 12 01:58:46.644072 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:46.644041    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"]
Sep 12 01:58:46.644613 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:46.644274    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk" podUID="9f8e77ad-54e2-4d0d-aea2-24b8f41787dd" containerName="payload-pre-processing" containerID="cri-o://0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab" gracePeriod=30
Sep 12 01:58:46.859545 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:46.859505    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"]
Sep 12 01:58:46.862981 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:46.862811    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-57c4f4a4-k2gbg"]
Sep 12 01:58:46.946345 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:46.946322    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:47.009235 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.004060    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e290a594-43d4-4b2f-9a18-01ddc4d24600" path="/var/lib/kubelet/pods/e290a594-43d4-4b2f-9a18-01ddc4d24600/volumes"
Sep 12 01:58:47.016227 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.016183    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"]
Sep 12 01:58:47.020229 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.016541    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy" containerID="cri-o://a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015" gracePeriod=30
Sep 12 01:58:47.046837 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.046801    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-plugins-config-volume\") pod \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\" (UID: \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\") "
Sep 12 01:58:47.046954 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.046925    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-kube-api-access-sm94h\") pod \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\" (UID: \"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd\") "
Sep 12 01:58:47.047415 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.047392    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f8e77ad-54e2-4d0d-aea2-24b8f41787dd" (UID: "9f8e77ad-54e2-4d0d-aea2-24b8f41787dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:58:47.049046 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.049021    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-kube-api-access-sm94h" (OuterVolumeSpecName: "kube-api-access-sm94h") pod "9f8e77ad-54e2-4d0d-aea2-24b8f41787dd" (UID: "9f8e77ad-54e2-4d0d-aea2-24b8f41787dd"). InnerVolumeSpecName "kube-api-access-sm94h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:58:47.148232 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.148180    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-kube-api-access-sm94h\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:47.148232 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.148236    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:47.501940 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.501904    2580 generic.go:356] "Generic (PLEG): container finished" podID="9f8e77ad-54e2-4d0d-aea2-24b8f41787dd" containerID="0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab" exitCode=0
Sep 12 01:58:47.502110 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.501990    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk" event={"ID":"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd","Type":"ContainerDied","Data":"0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab"}
Sep 12 01:58:47.502110 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.502020    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"
Sep 12 01:58:47.502110 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.502060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk" event={"ID":"9f8e77ad-54e2-4d0d-aea2-24b8f41787dd","Type":"ContainerDied","Data":"b51d8a659b6ceae3acb748d8058d3f862d07d6e2895f1fab83fe851ffbe64879"}
Sep 12 01:58:47.502110 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.502085    2580 scope.go:117] "RemoveContainer" containerID="0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab"
Sep 12 01:58:47.512721 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.512700    2580 scope.go:117] "RemoveContainer" containerID="0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab"
Sep 12 01:58:47.513043 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:58:47.513024    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab\": container with ID starting with 0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab not found: ID does not exist" containerID="0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab"
Sep 12 01:58:47.513122 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.513056    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab"} err="failed to get container status \"0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab\": rpc error: code = NotFound desc = could not find container \"0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab\": container with ID starting with 0f281e618175c3231e48fc56d635cbaa283f0a0c5e59abb2c57855aba5d35cab not found: ID does not exist"
Sep 12 01:58:47.530297 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.530272    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"]
Sep 12 01:58:47.540032 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:47.540008    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-57c4f4a4-545764d9cd-qcdhk"]
Sep 12 01:58:48.197237 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.196186    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"]
Sep 12 01:58:48.197237 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.196840    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e290a594-43d4-4b2f-9a18-01ddc4d24600" containerName="revoke-keys"
Sep 12 01:58:48.197237 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.196862    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e290a594-43d4-4b2f-9a18-01ddc4d24600" containerName="revoke-keys"
Sep 12 01:58:48.197237 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.196891    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8e77ad-54e2-4d0d-aea2-24b8f41787dd" containerName="payload-pre-processing"
Sep 12 01:58:48.197237 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.196900    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8e77ad-54e2-4d0d-aea2-24b8f41787dd" containerName="payload-pre-processing"
Sep 12 01:58:48.197237 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.197003    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8e77ad-54e2-4d0d-aea2-24b8f41787dd" containerName="payload-pre-processing"
Sep 12 01:58:48.197237 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.197017    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e290a594-43d4-4b2f-9a18-01ddc4d24600" containerName="revoke-keys"
Sep 12 01:58:48.200115 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.200093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.204448 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.204420    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-899a9ad9-gw-openshift-default-dockercfg-2sx95\""
Sep 12 01:58:48.225534 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.225507    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"]
Sep 12 01:58:48.258141 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.258103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-credential-socket\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.258458 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.258437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-data\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.258631 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.258618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-podinfo\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.258723 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.258712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7qnm\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-kube-api-access-w7qnm\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.258814 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.258803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-token\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.258895 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.258883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-certs\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.258984 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.258974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-socket\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.259082 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.259072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-envoy\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.259180 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.259170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istiod-ca-cert\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360135 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-podinfo\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360135 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7qnm\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-kube-api-access-w7qnm\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360407 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-token\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360407 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-certs\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360407 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-socket\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360562 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-envoy\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360676 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-socket\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360676 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istiod-ca-cert\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360849 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-certs\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360849 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-credential-socket\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.360849 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.360782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-data\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.361044 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.361021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-credential-socket\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.361134 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.361115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-data\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.361494 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.361462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istiod-ca-cert\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.362650 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.362622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-podinfo\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.362760 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.362741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-envoy\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.374881 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.374847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7qnm\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-kube-api-access-w7qnm\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.376693 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.375770    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-token\") pod \"e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") " pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.511086 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.511055    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:48.807381 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.807353    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"]
Sep 12 01:58:48.808678 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:48.808653    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d42f314_ae27_4d8e_8273_0eb25f6090a4.slice/crio-1e50600b63f4c028736615f0a4b6ae2d6f26f3584f985167485d6b46f54042a5 WatchSource:0}: Error finding container 1e50600b63f4c028736615f0a4b6ae2d6f26f3584f985167485d6b46f54042a5: Status 404 returned error can't find the container with id 1e50600b63f4c028736615f0a4b6ae2d6f26f3584f985167485d6b46f54042a5
Sep 12 01:58:48.811086 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.811052    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:48.811170 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.811138    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:48.811170 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.811167    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:48.984987 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:48.984944    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8e77ad-54e2-4d0d-aea2-24b8f41787dd" path="/var/lib/kubelet/pods/9f8e77ad-54e2-4d0d-aea2-24b8f41787dd/volumes"
Sep 12 01:58:49.518121 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:49.518086    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" event={"ID":"8d42f314-ae27-4d8e-8273-0eb25f6090a4","Type":"ContainerStarted","Data":"39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736"}
Sep 12 01:58:49.518530 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:49.518144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" event={"ID":"8d42f314-ae27-4d8e-8273-0eb25f6090a4","Type":"ContainerStarted","Data":"1e50600b63f4c028736615f0a4b6ae2d6f26f3584f985167485d6b46f54042a5"}
Sep 12 01:58:49.547760 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:49.547707    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" podStartSLOduration=1.547689358 podStartE2EDuration="1.547689358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:49.545947503 +0000 UTC m=+835.072848617" watchObservedRunningTime="2026-09-12 01:58:49.547689358 +0000 UTC m=+835.074590438"
Sep 12 01:58:50.512031 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:50.511989    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:50.517339 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:50.517307    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:50.523937 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:50.523887    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:50.524850 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:50.524831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:58:52.296240 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.296146    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:52.413455 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413414    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96nj9\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-kube-api-access-96nj9\") pod \"07996748-3c28-4ad3-8865-ed7073d88a42\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") "
Sep 12 01:58:52.413640 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413471    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-envoy\") pod \"07996748-3c28-4ad3-8865-ed7073d88a42\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") "
Sep 12 01:58:52.413640 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413582    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-socket\") pod \"07996748-3c28-4ad3-8865-ed7073d88a42\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") "
Sep 12 01:58:52.413778 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413646    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-data\") pod \"07996748-3c28-4ad3-8865-ed7073d88a42\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") "
Sep 12 01:58:52.413778 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413674    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-credential-socket\") pod \"07996748-3c28-4ad3-8865-ed7073d88a42\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") "
Sep 12 01:58:52.413778 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413730    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07996748-3c28-4ad3-8865-ed7073d88a42-istio-podinfo\") pod \"07996748-3c28-4ad3-8865-ed7073d88a42\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") "
Sep 12 01:58:52.413778 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413776    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-certs\") pod \"07996748-3c28-4ad3-8865-ed7073d88a42\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") "
Sep 12 01:58:52.413987 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413852    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07996748-3c28-4ad3-8865-ed7073d88a42" (UID: "07996748-3c28-4ad3-8865-ed7073d88a42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:58:52.413987 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413925    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-data" (OuterVolumeSpecName: "istio-data") pod "07996748-3c28-4ad3-8865-ed7073d88a42" (UID: "07996748-3c28-4ad3-8865-ed7073d88a42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:58:52.413987 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.413954    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07996748-3c28-4ad3-8865-ed7073d88a42" (UID: "07996748-3c28-4ad3-8865-ed7073d88a42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:58:52.414141 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.414070    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07996748-3c28-4ad3-8865-ed7073d88a42" (UID: "07996748-3c28-4ad3-8865-ed7073d88a42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:58:52.414200 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.414173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07996748-3c28-4ad3-8865-ed7073d88a42-istiod-ca-cert\") pod \"07996748-3c28-4ad3-8865-ed7073d88a42\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") "
Sep 12 01:58:52.414279 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.414235    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-istio-token\") pod \"07996748-3c28-4ad3-8865-ed7073d88a42\" (UID: \"07996748-3c28-4ad3-8865-ed7073d88a42\") "
Sep 12 01:58:52.414336 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.414290    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07996748-3c28-4ad3-8865-ed7073d88a42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07996748-3c28-4ad3-8865-ed7073d88a42" (UID: "07996748-3c28-4ad3-8865-ed7073d88a42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:58:52.414586 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.414565    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:52.414586 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.414586    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:52.414715 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.414595    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:52.414715 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.414605    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:52.414715 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.414614    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07996748-3c28-4ad3-8865-ed7073d88a42-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:52.415923 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.415891    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-kube-api-access-96nj9" (OuterVolumeSpecName: "kube-api-access-96nj9") pod "07996748-3c28-4ad3-8865-ed7073d88a42" (UID: "07996748-3c28-4ad3-8865-ed7073d88a42"). InnerVolumeSpecName "kube-api-access-96nj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:58:52.416513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.416483    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07996748-3c28-4ad3-8865-ed7073d88a42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07996748-3c28-4ad3-8865-ed7073d88a42" (UID: "07996748-3c28-4ad3-8865-ed7073d88a42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 01:58:52.416513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.416488    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07996748-3c28-4ad3-8865-ed7073d88a42" (UID: "07996748-3c28-4ad3-8865-ed7073d88a42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:58:52.416912 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.416897    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-istio-token" (OuterVolumeSpecName: "istio-token") pod "07996748-3c28-4ad3-8865-ed7073d88a42" (UID: "07996748-3c28-4ad3-8865-ed7073d88a42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:58:52.516070 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.516027    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07996748-3c28-4ad3-8865-ed7073d88a42-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:52.516070 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.516061    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:52.516070 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.516074    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96nj9\" (UniqueName: \"kubernetes.io/projected/07996748-3c28-4ad3-8865-ed7073d88a42-kube-api-access-96nj9\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:52.516342 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.516086    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07996748-3c28-4ad3-8865-ed7073d88a42-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:58:52.538991 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.538960    2580 generic.go:356] "Generic (PLEG): container finished" podID="07996748-3c28-4ad3-8865-ed7073d88a42" containerID="a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015" exitCode=0
Sep 12 01:58:52.539125 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.539010    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"
Sep 12 01:58:52.539125 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.539057    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" event={"ID":"07996748-3c28-4ad3-8865-ed7073d88a42","Type":"ContainerDied","Data":"a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015"}
Sep 12 01:58:52.539232 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.539127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f" event={"ID":"07996748-3c28-4ad3-8865-ed7073d88a42","Type":"ContainerDied","Data":"6dcd8031477f44423a66b8dcd0e09d57d270655ba4168e9087137bf2a529121b"}
Sep 12 01:58:52.539232 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.539168    2580 scope.go:117] "RemoveContainer" containerID="a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015"
Sep 12 01:58:52.550323 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.550306    2580 scope.go:117] "RemoveContainer" containerID="3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306"
Sep 12 01:58:52.560001 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.559976    2580 scope.go:117] "RemoveContainer" containerID="a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015"
Sep 12 01:58:52.560359 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:58:52.560340    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015\": container with ID starting with a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015 not found: ID does not exist" containerID="a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015"
Sep 12 01:58:52.560420 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.560368    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015"} err="failed to get container status \"a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015\": rpc error: code = NotFound desc = could not find container \"a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015\": container with ID starting with a8dc0e97acaab49ed6ca0ccbdb93ebc6ecf39bea49f403febcf7b22f95f9c015 not found: ID does not exist"
Sep 12 01:58:52.560420 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.560385    2580 scope.go:117] "RemoveContainer" containerID="3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306"
Sep 12 01:58:52.560680 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:58:52.560657    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306\": container with ID starting with 3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306 not found: ID does not exist" containerID="3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306"
Sep 12 01:58:52.560729 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.560688    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306"} err="failed to get container status \"3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306\": rpc error: code = NotFound desc = could not find container \"3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306\": container with ID starting with 3eefaa3ab02317802f1bc10bc3c3a7609929ffe58190a316527c4b047adc0306 not found: ID does not exist"
Sep 12 01:58:52.566628 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.566600    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"]
Sep 12 01:58:52.575912 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.575888    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-57c4f4a4-openshift-default-797689c9f6-qfg5f"]
Sep 12 01:58:52.980372 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:52.980296    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" path="/var/lib/kubelet/pods/07996748-3c28-4ad3-8865-ed7073d88a42/volumes"
Sep 12 01:58:54.554847 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:54.554763    2580 generic.go:356] "Generic (PLEG): container finished" podID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerID="d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5" exitCode=0
Sep 12 01:58:54.554847 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:54.554836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" event={"ID":"b17d3b35-603b-465c-aed7-e0ee82447dda","Type":"ContainerDied","Data":"d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5"}
Sep 12 01:58:54.555359 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:54.555339    2580 scope.go:117] "RemoveContainer" containerID="d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5"
Sep 12 01:58:54.556525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:54.556494    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:54.556605 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:54.556590    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:54.556645 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:54.556637    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:58:55.471513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.471430    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"]
Sep 12 01:58:55.472061 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.472045    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy"
Sep 12 01:58:55.472103 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.472066    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy"
Sep 12 01:58:55.472103 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.472088    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy"
Sep 12 01:58:55.472103 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.472097    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy"
Sep 12 01:58:55.472232 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.472198    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy"
Sep 12 01:58:55.476088 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.476068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:58:55.479075 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.478836    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-899a9ad9\""
Sep 12 01:58:55.479867 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.479847    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-899a9ad9-dockercfg-9mhwl\""
Sep 12 01:58:55.484893 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.484337    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"]
Sep 12 01:58:55.499776 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.499748    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"]
Sep 12 01:58:55.500291 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.500269    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07996748-3c28-4ad3-8865-ed7073d88a42" containerName="istio-proxy"
Sep 12 01:58:55.502547 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.502524    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:58:55.515548 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.515523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"]
Sep 12 01:58:55.545267 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.545240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v5sf\" (UniqueName: \"kubernetes.io/projected/e6110e97-3581-450a-98ab-25385cebf31b-kube-api-access-2v5sf\") pod \"payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh\" (UID: \"e6110e97-3581-450a-98ab-25385cebf31b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:58:55.545412 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.545285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6110e97-3581-450a-98ab-25385cebf31b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh\" (UID: \"e6110e97-3581-450a-98ab-25385cebf31b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:58:55.589049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.588999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" event={"ID":"b17d3b35-603b-465c-aed7-e0ee82447dda","Type":"ContainerStarted","Data":"a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709"}
Sep 12 01:58:55.646753 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.646720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-plugins-config-volume\") pod \"payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8\" (UID: \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\") " pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:58:55.646945 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.646789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v5sf\" (UniqueName: \"kubernetes.io/projected/e6110e97-3581-450a-98ab-25385cebf31b-kube-api-access-2v5sf\") pod \"payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh\" (UID: \"e6110e97-3581-450a-98ab-25385cebf31b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:58:55.646945 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.646866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wjbs\" (UniqueName: \"kubernetes.io/projected/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-kube-api-access-7wjbs\") pod \"payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8\" (UID: \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\") " pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:58:55.646945 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.646907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6110e97-3581-450a-98ab-25385cebf31b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh\" (UID: \"e6110e97-3581-450a-98ab-25385cebf31b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:58:55.647647 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.647627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6110e97-3581-450a-98ab-25385cebf31b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh\" (UID: \"e6110e97-3581-450a-98ab-25385cebf31b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:58:55.655160 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.655140    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v5sf\" (UniqueName: \"kubernetes.io/projected/e6110e97-3581-450a-98ab-25385cebf31b-kube-api-access-2v5sf\") pod \"payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh\" (UID: \"e6110e97-3581-450a-98ab-25385cebf31b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:58:55.746245 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.746217    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:58:55.747611 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.747582    2580 patch_prober.go:28] interesting pod/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 01:58:55.747747 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.747620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wjbs\" (UniqueName: \"kubernetes.io/projected/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-kube-api-access-7wjbs\") pod \"payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8\" (UID: \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\") " pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:58:55.747747 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.747640    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 01:58:55.747950 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.747921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-plugins-config-volume\") pod \"payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8\" (UID: \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\") " pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:58:55.748698 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.748680    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-plugins-config-volume\") pod \"payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8\" (UID: \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\") " pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:58:55.758353 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.758324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wjbs\" (UniqueName: \"kubernetes.io/projected/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-kube-api-access-7wjbs\") pod \"payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8\" (UID: \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\") " pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:58:55.788034 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.788001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:58:55.814780 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:55.814751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:58:56.074373 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:56.074341    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6110e97_3581_450a_98ab_25385cebf31b.slice/crio-4512e006898b7e1290bf0cfea559a9fc84ee95d9b625766a1a506a1d8e3be04f WatchSource:0}: Error finding container 4512e006898b7e1290bf0cfea559a9fc84ee95d9b625766a1a506a1d8e3be04f: Status 404 returned error can't find the container with id 4512e006898b7e1290bf0cfea559a9fc84ee95d9b625766a1a506a1d8e3be04f
Sep 12 01:58:56.074722 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.074694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"]
Sep 12 01:58:56.107887 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.107865    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"]
Sep 12 01:58:56.109790 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:58:56.109770    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee34cd7b_3edc_4545_b5b0_43b3ccf51d58.slice/crio-e6c0f55fd72322bd73374fd5be479cb98690ede55ef383aa8e1d93c2652cc06a WatchSource:0}: Error finding container e6c0f55fd72322bd73374fd5be479cb98690ede55ef383aa8e1d93c2652cc06a: Status 404 returned error can't find the container with id e6c0f55fd72322bd73374fd5be479cb98690ede55ef383aa8e1d93c2652cc06a
Sep 12 01:58:56.600910 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.600435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8" event={"ID":"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58","Type":"ContainerStarted","Data":"0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53"}
Sep 12 01:58:56.600910 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.600508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8" event={"ID":"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58","Type":"ContainerStarted","Data":"e6c0f55fd72322bd73374fd5be479cb98690ede55ef383aa8e1d93c2652cc06a"}
Sep 12 01:58:56.600910 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.600563    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:58:56.604264 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.603873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh" event={"ID":"e6110e97-3581-450a-98ab-25385cebf31b","Type":"ContainerStarted","Data":"d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b"}
Sep 12 01:58:56.604264 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.603941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh" event={"ID":"e6110e97-3581-450a-98ab-25385cebf31b","Type":"ContainerStarted","Data":"4512e006898b7e1290bf0cfea559a9fc84ee95d9b625766a1a506a1d8e3be04f"}
Sep 12 01:58:56.619260 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.619176    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8" podStartSLOduration=1.619157313 podStartE2EDuration="1.619157313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:56.618660667 +0000 UTC m=+842.145561749" watchObservedRunningTime="2026-09-12 01:58:56.619157313 +0000 UTC m=+842.146058394"
Sep 12 01:58:56.637039 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.636994    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh" podStartSLOduration=1.636979303 podStartE2EDuration="1.636979303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:58:56.635932042 +0000 UTC m=+842.162833114" watchObservedRunningTime="2026-09-12 01:58:56.636979303 +0000 UTC m=+842.163880374"
Sep 12 01:58:56.746652 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.746615    2580 patch_prober.go:28] interesting pod/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 01:58:56.746878 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:56.746673    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 01:58:57.610403 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:57.610372    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:58:57.745841 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:57.745804    2580 patch_prober.go:28] interesting pod/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 01:58:57.746042 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:57.745872    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 01:58:58.746611 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:58.746566    2580 patch_prober.go:28] interesting pod/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 01:58:58.747008 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:58.746637    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 01:58:59.746539 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:59.746500    2580 patch_prober.go:28] interesting pod/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 01:58:59.746726 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:58:59.746565    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 01:59:00.314275 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:00.314235    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:59:00.746437 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:00.746408    2580 patch_prober.go:28] interesting pod/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 01:59:00.746598 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:00.746462    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 01:59:01.745942 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:01.745902    2580 patch_prober.go:28] interesting pod/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 12 01:59:01.746364 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:01.745966    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 12 01:59:02.749642 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:02.749610    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:59:02.750761 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:02.750737    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 01:59:07.612828 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:07.612794    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:59:08.619161 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:08.619132    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:59:15.748831 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:15.748798    2580 generic.go:356] "Generic (PLEG): container finished" podID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerID="39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736" exitCode=0
Sep 12 01:59:15.749341 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:15.748881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" event={"ID":"8d42f314-ae27-4d8e-8273-0eb25f6090a4","Type":"ContainerDied","Data":"39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736"}
Sep 12 01:59:15.749410 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:15.749385    2580 scope.go:117] "RemoveContainer" containerID="39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736"
Sep 12 01:59:15.750541 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:15.750507    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:59:15.750657 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:15.750598    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:59:15.750657 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:15.750649    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:59:16.757058 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:16.757017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" event={"ID":"8d42f314-ae27-4d8e-8273-0eb25f6090a4","Type":"ContainerStarted","Data":"8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a"}
Sep 12 01:59:17.511931 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:17.511889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:59:17.513223 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:17.513178    2580 patch_prober.go:28] interesting pod/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 12 01:59:17.513301 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:17.513261    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 12 01:59:18.512293 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:18.512251    2580 patch_prober.go:28] interesting pod/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 12 01:59:18.512703 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:18.512320    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 12 01:59:19.512262 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:19.512227    2580 patch_prober.go:28] interesting pod/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 12 01:59:19.512466 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:19.512301    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 12 01:59:20.511779 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:20.511734    2580 patch_prober.go:28] interesting pod/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 12 01:59:20.511982 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:20.511799    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 12 01:59:20.524508 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:20.524476    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:59:21.182339 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.182301    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"]
Sep 12 01:59:21.185954 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.185930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"
Sep 12 01:59:21.191011 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.190986    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 01:59:21.191515 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.191493    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 01:59:21.195757 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.195730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"]
Sep 12 01:59:21.325580 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.325542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c3309dc-ccbe-4f9d-9f9c-62487399eafe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc\" (UID: \"6c3309dc-ccbe-4f9d-9f9c-62487399eafe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"
Sep 12 01:59:21.426259 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.426195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c3309dc-ccbe-4f9d-9f9c-62487399eafe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc\" (UID: \"6c3309dc-ccbe-4f9d-9f9c-62487399eafe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"
Sep 12 01:59:21.426831 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.426809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c3309dc-ccbe-4f9d-9f9c-62487399eafe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc\" (UID: \"6c3309dc-ccbe-4f9d-9f9c-62487399eafe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"
Sep 12 01:59:21.506074 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.506044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"
Sep 12 01:59:21.511382 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.511357    2580 patch_prober.go:28] interesting pod/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 12 01:59:21.511447 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.511406    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 12 01:59:21.779613 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.779567    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"]
Sep 12 01:59:21.780637 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:59:21.780597    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c3309dc_ccbe_4f9d_9f9c_62487399eafe.slice/crio-f42795139312a2a283588f4644d699cff163183d99c2841892c4ddc89fe02f0b WatchSource:0}: Error finding container f42795139312a2a283588f4644d699cff163183d99c2841892c4ddc89fe02f0b: Status 404 returned error can't find the container with id f42795139312a2a283588f4644d699cff163183d99c2841892c4ddc89fe02f0b
Sep 12 01:59:21.795790 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:21.795755    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc" event={"ID":"6c3309dc-ccbe-4f9d-9f9c-62487399eafe","Type":"ContainerStarted","Data":"f42795139312a2a283588f4644d699cff163183d99c2841892c4ddc89fe02f0b"}
Sep 12 01:59:22.515863 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:22.515832    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:59:22.516843 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:22.516825    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:59:22.803066 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:22.802974    2580 generic.go:356] "Generic (PLEG): container finished" podID="6c3309dc-ccbe-4f9d-9f9c-62487399eafe" containerID="791b6d960402d0d41bac3607fa209fa3e7910dcedd9c2c9e804f55b247708251" exitCode=0
Sep 12 01:59:22.803453 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:22.803065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc" event={"ID":"6c3309dc-ccbe-4f9d-9f9c-62487399eafe","Type":"ContainerDied","Data":"791b6d960402d0d41bac3607fa209fa3e7910dcedd9c2c9e804f55b247708251"}
Sep 12 01:59:23.953139 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:23.953114    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"
Sep 12 01:59:24.055708 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:24.055673    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c3309dc-ccbe-4f9d-9f9c-62487399eafe-maas-api-service-ca\") pod \"6c3309dc-ccbe-4f9d-9f9c-62487399eafe\" (UID: \"6c3309dc-ccbe-4f9d-9f9c-62487399eafe\") "
Sep 12 01:59:24.056053 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:24.056029    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c3309dc-ccbe-4f9d-9f9c-62487399eafe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6c3309dc-ccbe-4f9d-9f9c-62487399eafe" (UID: "6c3309dc-ccbe-4f9d-9f9c-62487399eafe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:59:24.157155 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:24.157054    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c3309dc-ccbe-4f9d-9f9c-62487399eafe-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:24.816494 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:24.816457    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"
Sep 12 01:59:24.816705 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:24.816463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc" event={"ID":"6c3309dc-ccbe-4f9d-9f9c-62487399eafe","Type":"ContainerDied","Data":"f42795139312a2a283588f4644d699cff163183d99c2841892c4ddc89fe02f0b"}
Sep 12 01:59:24.816705 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:24.816587    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f42795139312a2a283588f4644d699cff163183d99c2841892c4ddc89fe02f0b"
Sep 12 01:59:32.478659 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.478614    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"]
Sep 12 01:59:32.479103 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.478870    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8" podUID="ee34cd7b-3edc-4545-b5b0-43b3ccf51d58" containerName="payload-processing" containerID="cri-o://0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53" gracePeriod=30
Sep 12 01:59:32.486537 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.486505    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"]
Sep 12 01:59:32.486797 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.486741    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh" podUID="e6110e97-3581-450a-98ab-25385cebf31b" containerName="payload-pre-processing" containerID="cri-o://d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b" gracePeriod=30
Sep 12 01:59:32.634442 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.634385    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"]
Sep 12 01:59:32.637093 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.637067    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-899a9ad9-cq6fc"]
Sep 12 01:59:32.773239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.771792    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"]
Sep 12 01:59:32.773239 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.772042    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy" containerID="cri-o://8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a" gracePeriod=30
Sep 12 01:59:32.813475 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.813444    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:59:32.816448 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.816426    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:59:32.878462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.871278    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee34cd7b-3edc-4545-b5b0-43b3ccf51d58" containerID="0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53" exitCode=0
Sep 12 01:59:32.878462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.872789    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"
Sep 12 01:59:32.878462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.873500    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8" event={"ID":"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58","Type":"ContainerDied","Data":"0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53"}
Sep 12 01:59:32.878462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.873554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8" event={"ID":"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58","Type":"ContainerDied","Data":"e6c0f55fd72322bd73374fd5be479cb98690ede55ef383aa8e1d93c2652cc06a"}
Sep 12 01:59:32.878462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.873584    2580 scope.go:117] "RemoveContainer" containerID="0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53"
Sep 12 01:59:32.878462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.877636    2580 generic.go:356] "Generic (PLEG): container finished" podID="e6110e97-3581-450a-98ab-25385cebf31b" containerID="d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b" exitCode=0
Sep 12 01:59:32.878462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.877821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh" event={"ID":"e6110e97-3581-450a-98ab-25385cebf31b","Type":"ContainerDied","Data":"d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b"}
Sep 12 01:59:32.878462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.877878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh" event={"ID":"e6110e97-3581-450a-98ab-25385cebf31b","Type":"ContainerDied","Data":"4512e006898b7e1290bf0cfea559a9fc84ee95d9b625766a1a506a1d8e3be04f"}
Sep 12 01:59:32.878462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.877962    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"
Sep 12 01:59:32.896404 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.896371    2580 scope.go:117] "RemoveContainer" containerID="0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53"
Sep 12 01:59:32.896941 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:59:32.896904    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53\": container with ID starting with 0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53 not found: ID does not exist" containerID="0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53"
Sep 12 01:59:32.897061 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.896953    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53"} err="failed to get container status \"0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53\": rpc error: code = NotFound desc = could not find container \"0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53\": container with ID starting with 0cef3cecc095d1fa659110ddaa56cbd147d952989ca4b8df660f93abf16ead53 not found: ID does not exist"
Sep 12 01:59:32.897061 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.896982    2580 scope.go:117] "RemoveContainer" containerID="d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b"
Sep 12 01:59:32.912819 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.912779    2580 scope.go:117] "RemoveContainer" containerID="d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b"
Sep 12 01:59:32.913286 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:59:32.913258    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b\": container with ID starting with d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b not found: ID does not exist" containerID="d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b"
Sep 12 01:59:32.913398 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.913298    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b"} err="failed to get container status \"d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b\": rpc error: code = NotFound desc = could not find container \"d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b\": container with ID starting with d87cb0cb9627de330086d2517ccc914c953d7fe1df3eb3ee62cea22b667c8f0b not found: ID does not exist"
Sep 12 01:59:32.943709 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.943676    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6110e97-3581-450a-98ab-25385cebf31b-plugins-config-volume\") pod \"e6110e97-3581-450a-98ab-25385cebf31b\" (UID: \"e6110e97-3581-450a-98ab-25385cebf31b\") "
Sep 12 01:59:32.943871 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.943768    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v5sf\" (UniqueName: \"kubernetes.io/projected/e6110e97-3581-450a-98ab-25385cebf31b-kube-api-access-2v5sf\") pod \"e6110e97-3581-450a-98ab-25385cebf31b\" (UID: \"e6110e97-3581-450a-98ab-25385cebf31b\") "
Sep 12 01:59:32.943942 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.943875    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-plugins-config-volume\") pod \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\" (UID: \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\") "
Sep 12 01:59:32.943942 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.943923    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wjbs\" (UniqueName: \"kubernetes.io/projected/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-kube-api-access-7wjbs\") pod \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\" (UID: \"ee34cd7b-3edc-4545-b5b0-43b3ccf51d58\") "
Sep 12 01:59:32.944310 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.944258    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee34cd7b-3edc-4545-b5b0-43b3ccf51d58" (UID: "ee34cd7b-3edc-4545-b5b0-43b3ccf51d58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:59:32.944310 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.944258    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6110e97-3581-450a-98ab-25385cebf31b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6110e97-3581-450a-98ab-25385cebf31b" (UID: "e6110e97-3581-450a-98ab-25385cebf31b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:59:32.944458 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.944404    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6110e97-3581-450a-98ab-25385cebf31b-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:32.944458 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.944426    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:32.945846 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.945826    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6110e97-3581-450a-98ab-25385cebf31b-kube-api-access-2v5sf" (OuterVolumeSpecName: "kube-api-access-2v5sf") pod "e6110e97-3581-450a-98ab-25385cebf31b" (UID: "e6110e97-3581-450a-98ab-25385cebf31b"). InnerVolumeSpecName "kube-api-access-2v5sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:59:32.946012 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.945999    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-kube-api-access-7wjbs" (OuterVolumeSpecName: "kube-api-access-7wjbs") pod "ee34cd7b-3edc-4545-b5b0-43b3ccf51d58" (UID: "ee34cd7b-3edc-4545-b5b0-43b3ccf51d58"). InnerVolumeSpecName "kube-api-access-7wjbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:59:32.980192 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:32.980158    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c3309dc-ccbe-4f9d-9f9c-62487399eafe" path="/var/lib/kubelet/pods/6c3309dc-ccbe-4f9d-9f9c-62487399eafe/volumes"
Sep 12 01:59:33.045118 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.045048    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v5sf\" (UniqueName: \"kubernetes.io/projected/e6110e97-3581-450a-98ab-25385cebf31b-kube-api-access-2v5sf\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:33.045118 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.045077    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wjbs\" (UniqueName: \"kubernetes.io/projected/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58-kube-api-access-7wjbs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:33.191569 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.191530    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"]
Sep 12 01:59:33.195950 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.195922    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-899a9ad9-595fddd458-m89l8"]
Sep 12 01:59:33.206462 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.206428    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"]
Sep 12 01:59:33.213228 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.209951    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-899a9ad9-546c87794-5lkfh"]
Sep 12 01:59:33.574983 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.574946    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"]
Sep 12 01:59:33.575630 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.575610    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6110e97-3581-450a-98ab-25385cebf31b" containerName="payload-pre-processing"
Sep 12 01:59:33.575684 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.575633    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6110e97-3581-450a-98ab-25385cebf31b" containerName="payload-pre-processing"
Sep 12 01:59:33.575684 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.575648    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee34cd7b-3edc-4545-b5b0-43b3ccf51d58" containerName="payload-processing"
Sep 12 01:59:33.575684 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.575658    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee34cd7b-3edc-4545-b5b0-43b3ccf51d58" containerName="payload-processing"
Sep 12 01:59:33.575795 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.575688    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c3309dc-ccbe-4f9d-9f9c-62487399eafe" containerName="revoke-keys"
Sep 12 01:59:33.575795 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.575697    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c3309dc-ccbe-4f9d-9f9c-62487399eafe" containerName="revoke-keys"
Sep 12 01:59:33.575795 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.575788    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c3309dc-ccbe-4f9d-9f9c-62487399eafe" containerName="revoke-keys"
Sep 12 01:59:33.575883 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.575803    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee34cd7b-3edc-4545-b5b0-43b3ccf51d58" containerName="payload-processing"
Sep 12 01:59:33.575883 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.575816    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6110e97-3581-450a-98ab-25385cebf31b" containerName="payload-pre-processing"
Sep 12 01:59:33.579230 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.579200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.581900 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.581878    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-29d2f735-openshift-default-dockercfg-9f9t7\""
Sep 12 01:59:33.589357 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.589336    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"]
Sep 12 01:59:33.752583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.752495    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-envoy\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.752583 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.752549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-credential-socket\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.752822 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.752598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q67rk\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-kube-api-access-q67rk\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.752822 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.752633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-token\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.752822 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.752719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-socket\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.752822 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.752780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istiod-ca-cert\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.753020 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.752851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-data\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.753020 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.752881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-podinfo\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.753020 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.752924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-certs\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.853958 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.853922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-token\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854130 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.853972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-socket\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854130 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istiod-ca-cert\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854130 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-data\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854130 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-podinfo\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854130 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-certs\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854130 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-envoy\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854491 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-credential-socket\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854491 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q67rk\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-kube-api-access-q67rk\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854491 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854471    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-socket\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854645 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-data\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854645 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-certs\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854781 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-credential-socket\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.854907 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.854879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istiod-ca-cert\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.856540 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.856522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-envoy\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.856774 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.856755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-podinfo\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.861632 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.861607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-token\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.861761 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.861745    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q67rk\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-kube-api-access-q67rk\") pod \"e2e-ait-29d2f735-openshift-default-69748fd8-sth5d\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") " pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:33.892367 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:33.892337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:34.187255 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:34.187220    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"]
Sep 12 01:59:34.188092 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:59:34.188071    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5315ed96_d7bc_4122_905e_7d7f3bfe9f5b.slice/crio-aa70c10a86134311014ec28fde4e53c5a98bee9b600a56c19e4518a76ba5422e WatchSource:0}: Error finding container aa70c10a86134311014ec28fde4e53c5a98bee9b600a56c19e4518a76ba5422e: Status 404 returned error can't find the container with id aa70c10a86134311014ec28fde4e53c5a98bee9b600a56c19e4518a76ba5422e
Sep 12 01:59:34.190470 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:34.190437    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:59:34.190552 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:34.190524    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:59:34.190588 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:34.190551    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:59:34.893777 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:34.893739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" event={"ID":"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b","Type":"ContainerStarted","Data":"09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0"}
Sep 12 01:59:34.894176 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:34.893796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" event={"ID":"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b","Type":"ContainerStarted","Data":"aa70c10a86134311014ec28fde4e53c5a98bee9b600a56c19e4518a76ba5422e"}
Sep 12 01:59:34.917138 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:34.917085    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" podStartSLOduration=1.91707093 podStartE2EDuration="1.91707093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:59:34.913645396 +0000 UTC m=+880.440546477" watchObservedRunningTime="2026-09-12 01:59:34.91707093 +0000 UTC m=+880.443972010"
Sep 12 01:59:34.980094 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:34.980063    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6110e97-3581-450a-98ab-25385cebf31b" path="/var/lib/kubelet/pods/e6110e97-3581-450a-98ab-25385cebf31b/volumes"
Sep 12 01:59:34.980508 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:34.980493    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee34cd7b-3edc-4545-b5b0-43b3ccf51d58" path="/var/lib/kubelet/pods/ee34cd7b-3edc-4545-b5b0-43b3ccf51d58/volumes"
Sep 12 01:59:35.892539 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:35.892506    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:35.898499 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:35.898462    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:35.903217 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:35.903190    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:35.903918 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:35.903901    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 01:59:37.229062 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.229022    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"]
Sep 12 01:59:37.231794 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.231776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.235358 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.235338    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-98f68e\"/\"openshift-service-ca.crt\""
Sep 12 01:59:37.235358 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.235350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-98f68e\"/\"simulator-w4-98f68e-kserve-self-signed-certs\""
Sep 12 01:59:37.235567 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.235394    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-98f68e\"/\"kube-root-ca.crt\""
Sep 12 01:59:37.235567 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.235354    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-98f68e\"/\"default-dockercfg-494kz\""
Sep 12 01:59:37.243361 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.243339    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"]
Sep 12 01:59:37.387660 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.387618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-dshm\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.387660 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.387659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb4343d-c86f-4c11-a279-1e29fec6f992-tls-certs\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.387899 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.387732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-model-cache\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.387899 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.387757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-home\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.387899 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.387830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwzfb\" (UniqueName: \"kubernetes.io/projected/3fb4343d-c86f-4c11-a279-1e29fec6f992-kube-api-access-zwzfb\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.388093 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.387912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-tmp-dir\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.488716 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.488638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-dshm\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.488716 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.488676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb4343d-c86f-4c11-a279-1e29fec6f992-tls-certs\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.488897 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.488740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-model-cache\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.488897 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.488763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-home\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.488897 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.488806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwzfb\" (UniqueName: \"kubernetes.io/projected/3fb4343d-c86f-4c11-a279-1e29fec6f992-kube-api-access-zwzfb\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.488897 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.488880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-tmp-dir\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.489188 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.489164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-home\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.489293 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.489274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-model-cache\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.489358 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.489275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-tmp-dir\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.490965 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.490939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-dshm\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.491245 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.491205    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb4343d-c86f-4c11-a279-1e29fec6f992-tls-certs\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.505068 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.505041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwzfb\" (UniqueName: \"kubernetes.io/projected/3fb4343d-c86f-4c11-a279-1e29fec6f992-kube-api-access-zwzfb\") pod \"simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") " pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.518612 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.518584    2580 patch_prober.go:28] interesting pod/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 12 01:59:37.518727 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.518649    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 01:59:37.542987 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.542959    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:37.832916 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.832884    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"]
Sep 12 01:59:37.834782 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:59:37.834751    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fb4343d_c86f_4c11_a279_1e29fec6f992.slice/crio-e275662405888cb302d0a2fea447815b40243434df1cf385db365f586adb1aee WatchSource:0}: Error finding container e275662405888cb302d0a2fea447815b40243434df1cf385db365f586adb1aee: Status 404 returned error can't find the container with id e275662405888cb302d0a2fea447815b40243434df1cf385db365f586adb1aee
Sep 12 01:59:37.917348 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.917310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp" event={"ID":"3fb4343d-c86f-4c11-a279-1e29fec6f992","Type":"ContainerStarted","Data":"ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701"}
Sep 12 01:59:37.917507 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.917362    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp" event={"ID":"3fb4343d-c86f-4c11-a279-1e29fec6f992","Type":"ContainerStarted","Data":"e275662405888cb302d0a2fea447815b40243434df1cf385db365f586adb1aee"}
Sep 12 01:59:37.938377 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:37.938325    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp" podStartSLOduration=0.93831001 podStartE2EDuration="938.31001ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:59:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:59:37.935650958 +0000 UTC m=+883.462552038" watchObservedRunningTime="2026-09-12 01:59:37.93831001 +0000 UTC m=+883.465211090"
Sep 12 01:59:38.141551 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.141460    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:59:38.299156 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299119    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-certs\") pod \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") "
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299165    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-data\") pod \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") "
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299192    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istiod-ca-cert\") pod \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") "
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299249    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7qnm\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-kube-api-access-w7qnm\") pod \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") "
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299268    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-envoy\") pod \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") "
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-podinfo\") pod \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") "
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299367    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-socket\") pod \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") "
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299442    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-credential-socket\") pod \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") "
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299490    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-token\") pod \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\" (UID: \"8d42f314-ae27-4d8e-8273-0eb25f6090a4\") "
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299515    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8d42f314-ae27-4d8e-8273-0eb25f6090a4" (UID: "8d42f314-ae27-4d8e-8273-0eb25f6090a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:59:38.299623 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299515    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-data" (OuterVolumeSpecName: "istio-data") pod "8d42f314-ae27-4d8e-8273-0eb25f6090a4" (UID: "8d42f314-ae27-4d8e-8273-0eb25f6090a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:59:38.300029 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299661    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8d42f314-ae27-4d8e-8273-0eb25f6090a4" (UID: "8d42f314-ae27-4d8e-8273-0eb25f6090a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:59:38.300029 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299639    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8d42f314-ae27-4d8e-8273-0eb25f6090a4" (UID: "8d42f314-ae27-4d8e-8273-0eb25f6090a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:59:38.300029 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299669    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8d42f314-ae27-4d8e-8273-0eb25f6090a4" (UID: "8d42f314-ae27-4d8e-8273-0eb25f6090a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:59:38.300029 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299824    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:38.300029 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299842    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:38.300029 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299858    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:38.300029 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299869    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:38.300029 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.299881    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:38.301715 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.301695    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8d42f314-ae27-4d8e-8273-0eb25f6090a4" (UID: "8d42f314-ae27-4d8e-8273-0eb25f6090a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:59:38.301799 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.301719    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8d42f314-ae27-4d8e-8273-0eb25f6090a4" (UID: "8d42f314-ae27-4d8e-8273-0eb25f6090a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 01:59:38.302024 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.302006    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d42f314-ae27-4d8e-8273-0eb25f6090a4" (UID: "8d42f314-ae27-4d8e-8273-0eb25f6090a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:59:38.302066 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.302013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-kube-api-access-w7qnm" (OuterVolumeSpecName: "kube-api-access-w7qnm") pod "8d42f314-ae27-4d8e-8273-0eb25f6090a4" (UID: "8d42f314-ae27-4d8e-8273-0eb25f6090a4"). InnerVolumeSpecName "kube-api-access-w7qnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:59:38.401050 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.400964    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:38.401050 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.400996    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7qnm\" (UniqueName: \"kubernetes.io/projected/8d42f314-ae27-4d8e-8273-0eb25f6090a4-kube-api-access-w7qnm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:38.401050 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.401005    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:38.401050 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.401014    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d42f314-ae27-4d8e-8273-0eb25f6090a4-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 01:59:38.720108 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.720029    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"]
Sep 12 01:59:38.720481 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.720467    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy"
Sep 12 01:59:38.720525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.720484    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy"
Sep 12 01:59:38.720525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.720508    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy"
Sep 12 01:59:38.720525 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.720514    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy"
Sep 12 01:59:38.720619 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.720580    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy"
Sep 12 01:59:38.720619 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.720591    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerName="istio-proxy"
Sep 12 01:59:38.722883 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.722867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:38.725465 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.725443    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-29d2f735-dockercfg-vs4vv\""
Sep 12 01:59:38.725615 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.725495    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-29d2f735\""
Sep 12 01:59:38.732050 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.732026    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"]
Sep 12 01:59:38.906532 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.906492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khj4p\" (UniqueName: \"kubernetes.io/projected/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-kube-api-access-khj4p\") pod \"payload-processing-e2e-ait-29d2f735-64df6f79-jjn28\" (UID: \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\") " pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:38.906692 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.906560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-plugins-config-volume\") pod \"payload-processing-e2e-ait-29d2f735-64df6f79-jjn28\" (UID: \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\") " pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:38.927064 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.927035    2580 generic.go:356] "Generic (PLEG): container finished" podID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" containerID="8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a" exitCode=0
Sep 12 01:59:38.927236 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.927082    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"
Sep 12 01:59:38.927236 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.927133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" event={"ID":"8d42f314-ae27-4d8e-8273-0eb25f6090a4","Type":"ContainerDied","Data":"8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a"}
Sep 12 01:59:38.927236 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.927198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv" event={"ID":"8d42f314-ae27-4d8e-8273-0eb25f6090a4","Type":"ContainerDied","Data":"1e50600b63f4c028736615f0a4b6ae2d6f26f3584f985167485d6b46f54042a5"}
Sep 12 01:59:38.927342 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.927246    2580 scope.go:117] "RemoveContainer" containerID="8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a"
Sep 12 01:59:38.937513 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.937493    2580 scope.go:117] "RemoveContainer" containerID="39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736"
Sep 12 01:59:38.946454 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.946433    2580 scope.go:117] "RemoveContainer" containerID="8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a"
Sep 12 01:59:38.946734 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:59:38.946712    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a\": container with ID starting with 8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a not found: ID does not exist" containerID="8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a"
Sep 12 01:59:38.946779 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.946743    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a"} err="failed to get container status \"8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a\": rpc error: code = NotFound desc = could not find container \"8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a\": container with ID starting with 8b1eac6c822645b0f8e5366cef55bb6a28560e7133269c684305c0fb9e0f215a not found: ID does not exist"
Sep 12 01:59:38.946779 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.946761    2580 scope.go:117] "RemoveContainer" containerID="39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736"
Sep 12 01:59:38.947204 ip-10-0-143-251 kubenswrapper[2580]: E0912 01:59:38.947167    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736\": container with ID starting with 39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736 not found: ID does not exist" containerID="39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736"
Sep 12 01:59:38.947340 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.947272    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736"} err="failed to get container status \"39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736\": rpc error: code = NotFound desc = could not find container \"39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736\": container with ID starting with 39d56d67bccea3f42df9aae17e7434a44e519172ce28a650b78be53dbbdda736 not found: ID does not exist"
Sep 12 01:59:38.950743 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.950702    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"]
Sep 12 01:59:38.953880 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.953860    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-899a9ad9-gw-openshift-default-5f4666c444-7cvqv"]
Sep 12 01:59:38.985649 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:38.985557    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d42f314-ae27-4d8e-8273-0eb25f6090a4" path="/var/lib/kubelet/pods/8d42f314-ae27-4d8e-8273-0eb25f6090a4/volumes"
Sep 12 01:59:39.007864 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.007827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khj4p\" (UniqueName: \"kubernetes.io/projected/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-kube-api-access-khj4p\") pod \"payload-processing-e2e-ait-29d2f735-64df6f79-jjn28\" (UID: \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\") " pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:39.008027 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.007889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-plugins-config-volume\") pod \"payload-processing-e2e-ait-29d2f735-64df6f79-jjn28\" (UID: \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\") " pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:39.008596 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.008570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-plugins-config-volume\") pod \"payload-processing-e2e-ait-29d2f735-64df6f79-jjn28\" (UID: \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\") " pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:39.016784 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.016760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khj4p\" (UniqueName: \"kubernetes.io/projected/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-kube-api-access-khj4p\") pod \"payload-processing-e2e-ait-29d2f735-64df6f79-jjn28\" (UID: \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\") " pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:39.034679 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.034644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:39.312514 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.312481    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"]
Sep 12 01:59:39.313921 ip-10-0-143-251 kubenswrapper[2580]: W0912 01:59:39.313897    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0da6dd7_e0d3_4d3a_9a5b_ceb66ac74d20.slice/crio-7f4a8dd44523c98913509f0b5351d601e937e93784e50887b37603da3c5ee11d WatchSource:0}: Error finding container 7f4a8dd44523c98913509f0b5351d601e937e93784e50887b37603da3c5ee11d: Status 404 returned error can't find the container with id 7f4a8dd44523c98913509f0b5351d601e937e93784e50887b37603da3c5ee11d
Sep 12 01:59:39.936772 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.936713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28" event={"ID":"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20","Type":"ContainerStarted","Data":"45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2"}
Sep 12 01:59:39.936981 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.936783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28" event={"ID":"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20","Type":"ContainerStarted","Data":"7f4a8dd44523c98913509f0b5351d601e937e93784e50887b37603da3c5ee11d"}
Sep 12 01:59:39.937049 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.937031    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:39.958386 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:39.958327    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28" podStartSLOduration=1.958312427 podStartE2EDuration="1.958312427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:59:39.955632929 +0000 UTC m=+885.482534008" watchObservedRunningTime="2026-09-12 01:59:39.958312427 +0000 UTC m=+885.485213506"
Sep 12 01:59:47.543728 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:47.543627    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:47.543728 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:47.543681    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:47.556578 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:47.556552    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:48.002643 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:48.002616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 01:59:50.947625 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:50.947588    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 01:59:55.011894 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:55.011866    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 01:59:55.015678 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:55.015655    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 01:59:59.064526 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:59.064491    2580 generic.go:356] "Generic (PLEG): container finished" podID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerID="09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0" exitCode=0
Sep 12 01:59:59.064958 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:59.064543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" event={"ID":"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b","Type":"ContainerDied","Data":"09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0"}
Sep 12 01:59:59.064958 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:59.064950    2580 scope.go:117] "RemoveContainer" containerID="09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0"
Sep 12 01:59:59.066117 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:59.066085    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:59:59.066257 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:59.066172    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 01:59:59.066257 ip-10-0-143-251 kubenswrapper[2580]: I0912 01:59:59.066239    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:00:00.085965 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.085922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" event={"ID":"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b","Type":"ContainerStarted","Data":"c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb"}
Sep 12 02:00:00.142237 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.142183    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn"]
Sep 12 02:00:00.145284 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.145247    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn"
Sep 12 02:00:00.148332 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.148311    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:00:00.156973 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.156943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn"
Sep 12 02:00:00.157108 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.156982    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs"]
Sep 12 02:00:00.159904 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.159882    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs"
Sep 12 02:00:00.163354 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.160857    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9"]
Sep 12 02:00:00.164825 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.164802    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7"]
Sep 12 02:00:00.165030 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.165013    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9"
Sep 12 02:00:00.179846 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.175019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7"
Sep 12 02:00:00.185395 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.185261    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn"]
Sep 12 02:00:00.220350 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.220320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs"
Sep 12 02:00:00.220556 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.220533    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs"]
Sep 12 02:00:00.227098 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.227063    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9"]
Sep 12 02:00:00.232775 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.232525    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9"
Sep 12 02:00:00.232775 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.232676    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7"]
Sep 12 02:00:00.316468 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.316254    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7"
Sep 12 02:00:00.517719 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.517693    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn"]
Sep 12 02:00:00.518785 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:00.518763    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7379ba0d_4e31_4f54_90dc_ce4d0bbc3dc0.slice/crio-5ce85b428737fd1dbbe98bd3a6ad186de07189e2ebfdff842b2e9c7f966883d4 WatchSource:0}: Error finding container 5ce85b428737fd1dbbe98bd3a6ad186de07189e2ebfdff842b2e9c7f966883d4: Status 404 returned error can't find the container with id 5ce85b428737fd1dbbe98bd3a6ad186de07189e2ebfdff842b2e9c7f966883d4
Sep 12 02:00:00.557821 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.557797    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9"]
Sep 12 02:00:00.558437 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:00.558406    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3dbb6b_d87f_41e7_826d_298cfae16122.slice/crio-7f4183b3fd85b5c6acc4a876211056b0e4220cff9b96a1c3057f412d4fd4ba94 WatchSource:0}: Error finding container 7f4183b3fd85b5c6acc4a876211056b0e4220cff9b96a1c3057f412d4fd4ba94: Status 404 returned error can't find the container with id 7f4183b3fd85b5c6acc4a876211056b0e4220cff9b96a1c3057f412d4fd4ba94
Sep 12 02:00:00.560613 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.560589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs"]
Sep 12 02:00:00.561188 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:00.561156    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ac7959_254c_4512_9917_1666ea8a04e2.slice/crio-f6631474b35292bc87553cb1a061cc3112e283e99eb72d563a486e61790db7b2 WatchSource:0}: Error finding container f6631474b35292bc87553cb1a061cc3112e283e99eb72d563a486e61790db7b2: Status 404 returned error can't find the container with id f6631474b35292bc87553cb1a061cc3112e283e99eb72d563a486e61790db7b2
Sep 12 02:00:00.639595 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.639571    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7"]
Sep 12 02:00:00.641094 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:00.641065    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod446090eb_e94f_4101_8876_791e5f3e63b9.slice/crio-c36ccba394bc82b41f5e4bdabf4513abdbff3021d268fa544b7ed81288b51153 WatchSource:0}: Error finding container c36ccba394bc82b41f5e4bdabf4513abdbff3021d268fa544b7ed81288b51153: Status 404 returned error can't find the container with id c36ccba394bc82b41f5e4bdabf4513abdbff3021d268fa544b7ed81288b51153
Sep 12 02:00:00.894324 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.893711    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 02:00:00.894324 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.893827    2580 patch_prober.go:28] interesting pod/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 12 02:00:00.894324 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:00.893866    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 12 02:00:01.099380 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.099062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9" event={"ID":"df3dbb6b-d87f-41e7-826d-298cfae16122","Type":"ContainerStarted","Data":"a2f907661400cc9392839ccfc6c46b9b51b063e7ce325fdbc7440da628f10a31"}
Sep 12 02:00:01.099380 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.099129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9" event={"ID":"df3dbb6b-d87f-41e7-826d-298cfae16122","Type":"ContainerStarted","Data":"7f4183b3fd85b5c6acc4a876211056b0e4220cff9b96a1c3057f412d4fd4ba94"}
Sep 12 02:00:01.101161 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.101127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn" event={"ID":"7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0","Type":"ContainerStarted","Data":"0a9c9f97a7e2eaeaac9b325c360552b233fcb9de54aa8300f36020bf64ff0ff8"}
Sep 12 02:00:01.101271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.101169    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn" event={"ID":"7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0","Type":"ContainerStarted","Data":"5ce85b428737fd1dbbe98bd3a6ad186de07189e2ebfdff842b2e9c7f966883d4"}
Sep 12 02:00:01.103188 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.103167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs" event={"ID":"71ac7959-254c-4512-9917-1666ea8a04e2","Type":"ContainerStarted","Data":"bbe84386087496f8e7981d1e273d0e55993a8d328dfb60e83ce6084aa494e6f7"}
Sep 12 02:00:01.103259 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.103221    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs" event={"ID":"71ac7959-254c-4512-9917-1666ea8a04e2","Type":"ContainerStarted","Data":"f6631474b35292bc87553cb1a061cc3112e283e99eb72d563a486e61790db7b2"}
Sep 12 02:00:01.104484 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.104462    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7" event={"ID":"446090eb-e94f-4101-8876-791e5f3e63b9","Type":"ContainerStarted","Data":"c36ccba394bc82b41f5e4bdabf4513abdbff3021d268fa544b7ed81288b51153"}
Sep 12 02:00:01.115756 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.115708    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9" podStartSLOduration=1.11569427 podStartE2EDuration="1.11569427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:00:01.113440862 +0000 UTC m=+906.640341937" watchObservedRunningTime="2026-09-12 02:00:01.11569427 +0000 UTC m=+906.642595350"
Sep 12 02:00:01.131634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.131539    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs" podStartSLOduration=1.131522695 podStartE2EDuration="1.131522695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:00:01.128373677 +0000 UTC m=+906.655274758" watchObservedRunningTime="2026-09-12 02:00:01.131522695 +0000 UTC m=+906.658423774"
Sep 12 02:00:01.144377 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.144328    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn" podStartSLOduration=1.1443145239999999 podStartE2EDuration="1.144314524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:00:01.141934865 +0000 UTC m=+906.668835944" watchObservedRunningTime="2026-09-12 02:00:01.144314524 +0000 UTC m=+906.671215656"
Sep 12 02:00:01.892995 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.892948    2580 patch_prober.go:28] interesting pod/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 12 02:00:01.893158 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:01.893025    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 12 02:00:02.113473 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.113437    2580 generic.go:356] "Generic (PLEG): container finished" podID="df3dbb6b-d87f-41e7-826d-298cfae16122" containerID="a2f907661400cc9392839ccfc6c46b9b51b063e7ce325fdbc7440da628f10a31" exitCode=0
Sep 12 02:00:02.113899 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.113517    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9" event={"ID":"df3dbb6b-d87f-41e7-826d-298cfae16122","Type":"ContainerDied","Data":"a2f907661400cc9392839ccfc6c46b9b51b063e7ce325fdbc7440da628f10a31"}
Sep 12 02:00:02.115197 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.115173    2580 generic.go:356] "Generic (PLEG): container finished" podID="7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0" containerID="0a9c9f97a7e2eaeaac9b325c360552b233fcb9de54aa8300f36020bf64ff0ff8" exitCode=0
Sep 12 02:00:02.115341 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.115223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn" event={"ID":"7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0","Type":"ContainerDied","Data":"0a9c9f97a7e2eaeaac9b325c360552b233fcb9de54aa8300f36020bf64ff0ff8"}
Sep 12 02:00:02.116692 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.116664    2580 generic.go:356] "Generic (PLEG): container finished" podID="71ac7959-254c-4512-9917-1666ea8a04e2" containerID="bbe84386087496f8e7981d1e273d0e55993a8d328dfb60e83ce6084aa494e6f7" exitCode=0
Sep 12 02:00:02.116771 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.116749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs" event={"ID":"71ac7959-254c-4512-9917-1666ea8a04e2","Type":"ContainerDied","Data":"bbe84386087496f8e7981d1e273d0e55993a8d328dfb60e83ce6084aa494e6f7"}
Sep 12 02:00:02.118267 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.118245    2580 generic.go:356] "Generic (PLEG): container finished" podID="446090eb-e94f-4101-8876-791e5f3e63b9" containerID="7aee1a8b0e89dac67e9eb496a4d507e10bf583b55e0795b16e93694e74072129" exitCode=0
Sep 12 02:00:02.118365 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.118302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7" event={"ID":"446090eb-e94f-4101-8876-791e5f3e63b9","Type":"ContainerDied","Data":"7aee1a8b0e89dac67e9eb496a4d507e10bf583b55e0795b16e93694e74072129"}
Sep 12 02:00:02.122064 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.122040    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"]
Sep 12 02:00:02.124998 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.124980    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"
Sep 12 02:00:02.127400 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.127384    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:00:02.142166 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.139556    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"]
Sep 12 02:00:02.256430 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.256385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/490c3b36-80dc-4afc-8fd9-6553aa35f041-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp\" (UID: \"490c3b36-80dc-4afc-8fd9-6553aa35f041\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"
Sep 12 02:00:02.357237 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.357118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/490c3b36-80dc-4afc-8fd9-6553aa35f041-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp\" (UID: \"490c3b36-80dc-4afc-8fd9-6553aa35f041\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"
Sep 12 02:00:02.357846 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.357819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/490c3b36-80dc-4afc-8fd9-6553aa35f041-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp\" (UID: \"490c3b36-80dc-4afc-8fd9-6553aa35f041\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"
Sep 12 02:00:02.446338 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.446292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"
Sep 12 02:00:02.750945 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.750917    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"]
Sep 12 02:00:02.752343 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:02.752309    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490c3b36_80dc_4afc_8fd9_6553aa35f041.slice/crio-dcb660f0d6ca9f55f1042598b77c6da81d1492aa9015274eb5181f27adec7189 WatchSource:0}: Error finding container dcb660f0d6ca9f55f1042598b77c6da81d1492aa9015274eb5181f27adec7189: Status 404 returned error can't find the container with id dcb660f0d6ca9f55f1042598b77c6da81d1492aa9015274eb5181f27adec7189
Sep 12 02:00:02.893608 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.893563    2580 patch_prober.go:28] interesting pod/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 12 02:00:02.893782 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:02.893649    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 12 02:00:03.126086 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:03.126048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp" event={"ID":"490c3b36-80dc-4afc-8fd9-6553aa35f041","Type":"ContainerStarted","Data":"dcb660f0d6ca9f55f1042598b77c6da81d1492aa9015274eb5181f27adec7189"}
Sep 12 02:00:03.333255 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:03.332841    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs"
Sep 12 02:00:03.363716 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:03.363683    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9"
Sep 12 02:00:03.417740 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:03.417704    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn"
Sep 12 02:00:03.420975 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:03.420950    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7"
Sep 12 02:00:03.893234 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:03.893169    2580 patch_prober.go:28] interesting pod/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 12 02:00:03.893435 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:03.893277    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 12 02:00:04.133642 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.133564    2580 generic.go:356] "Generic (PLEG): container finished" podID="490c3b36-80dc-4afc-8fd9-6553aa35f041" containerID="10dd848eb2811bc4d55548ccbf11addf97ffb602a4dbcfed5feca762cb4176e9" exitCode=0
Sep 12 02:00:04.134077 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.133648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp" event={"ID":"490c3b36-80dc-4afc-8fd9-6553aa35f041","Type":"ContainerDied","Data":"10dd848eb2811bc4d55548ccbf11addf97ffb602a4dbcfed5feca762cb4176e9"}
Sep 12 02:00:04.135494 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.135475    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7"
Sep 12 02:00:04.135796 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.135778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7" event={"ID":"446090eb-e94f-4101-8876-791e5f3e63b9","Type":"ContainerDied","Data":"c36ccba394bc82b41f5e4bdabf4513abdbff3021d268fa544b7ed81288b51153"}
Sep 12 02:00:04.135925 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.135910    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c36ccba394bc82b41f5e4bdabf4513abdbff3021d268fa544b7ed81288b51153"
Sep 12 02:00:04.137175 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.137158    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9"
Sep 12 02:00:04.137281 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.137247    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9" event={"ID":"df3dbb6b-d87f-41e7-826d-298cfae16122","Type":"ContainerDied","Data":"7f4183b3fd85b5c6acc4a876211056b0e4220cff9b96a1c3057f412d4fd4ba94"}
Sep 12 02:00:04.137356 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.137294    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f4183b3fd85b5c6acc4a876211056b0e4220cff9b96a1c3057f412d4fd4ba94"
Sep 12 02:00:04.139682 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.139668    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn"
Sep 12 02:00:04.139775 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.139708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn" event={"ID":"7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0","Type":"ContainerDied","Data":"5ce85b428737fd1dbbe98bd3a6ad186de07189e2ebfdff842b2e9c7f966883d4"}
Sep 12 02:00:04.139775 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.139737    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ce85b428737fd1dbbe98bd3a6ad186de07189e2ebfdff842b2e9c7f966883d4"
Sep 12 02:00:04.141237 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.141198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs" event={"ID":"71ac7959-254c-4512-9917-1666ea8a04e2","Type":"ContainerDied","Data":"f6631474b35292bc87553cb1a061cc3112e283e99eb72d563a486e61790db7b2"}
Sep 12 02:00:04.141367 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.141353    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6631474b35292bc87553cb1a061cc3112e283e99eb72d563a486e61790db7b2"
Sep 12 02:00:04.141461 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.141264    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs"
Sep 12 02:00:04.893358 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.893311    2580 patch_prober.go:28] interesting pod/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 12 02:00:04.893556 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:04.893382    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 12 02:00:05.312462 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:05.312427    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"
Sep 12 02:00:05.417242 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:05.417130    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/490c3b36-80dc-4afc-8fd9-6553aa35f041-maas-api-service-ca\") pod \"490c3b36-80dc-4afc-8fd9-6553aa35f041\" (UID: \"490c3b36-80dc-4afc-8fd9-6553aa35f041\") "
Sep 12 02:00:05.417564 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:05.417532    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490c3b36-80dc-4afc-8fd9-6553aa35f041-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "490c3b36-80dc-4afc-8fd9-6553aa35f041" (UID: "490c3b36-80dc-4afc-8fd9-6553aa35f041"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:00:05.518951 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:05.518917    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/490c3b36-80dc-4afc-8fd9-6553aa35f041-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:05.897449 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:05.897418    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 02:00:05.897727 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:05.897707    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 02:00:05.898528 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:05.898507    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 02:00:06.155467 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:06.155386    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"
Sep 12 02:00:06.155467 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:06.155419    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp" event={"ID":"490c3b36-80dc-4afc-8fd9-6553aa35f041","Type":"ContainerDied","Data":"dcb660f0d6ca9f55f1042598b77c6da81d1492aa9015274eb5181f27adec7189"}
Sep 12 02:00:06.155467 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:06.155462    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcb660f0d6ca9f55f1042598b77c6da81d1492aa9015274eb5181f27adec7189"
Sep 12 02:00:07.735880 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:07.735830    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn"]
Sep 12 02:00:07.738493 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:07.738469    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-29d2f735-29819640-l8kwn"]
Sep 12 02:00:07.771649 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:07.771608    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"]
Sep 12 02:00:07.771949 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:07.771918    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28" podUID="d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20" containerName="payload-processing" containerID="cri-o://45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2" gracePeriod=30
Sep 12 02:00:07.929928 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:07.929893    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"]
Sep 12 02:00:07.932923 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:07.932897    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-29d2f735-9n9cp"]
Sep 12 02:00:08.046429 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.046403    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 02:00:08.145132 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.145083    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-plugins-config-volume\") pod \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\" (UID: \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\") "
Sep 12 02:00:08.145376 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.145162    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khj4p\" (UniqueName: \"kubernetes.io/projected/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-kube-api-access-khj4p\") pod \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\" (UID: \"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20\") "
Sep 12 02:00:08.145593 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.145548    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20" (UID: "d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:00:08.147478 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.147444    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-kube-api-access-khj4p" (OuterVolumeSpecName: "kube-api-access-khj4p") pod "d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20" (UID: "d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20"). InnerVolumeSpecName "kube-api-access-khj4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:00:08.173818 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.173773    2580 generic.go:356] "Generic (PLEG): container finished" podID="d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20" containerID="45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2" exitCode=0
Sep 12 02:00:08.173977 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.173826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28" event={"ID":"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20","Type":"ContainerDied","Data":"45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2"}
Sep 12 02:00:08.173977 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.173876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28" event={"ID":"d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20","Type":"ContainerDied","Data":"7f4a8dd44523c98913509f0b5351d601e937e93784e50887b37603da3c5ee11d"}
Sep 12 02:00:08.173977 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.173905    2580 scope.go:117] "RemoveContainer" containerID="45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2"
Sep 12 02:00:08.174161 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.174057    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"
Sep 12 02:00:08.189123 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.189099    2580 scope.go:117] "RemoveContainer" containerID="45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2"
Sep 12 02:00:08.189503 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:00:08.189477    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2\": container with ID starting with 45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2 not found: ID does not exist" containerID="45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2"
Sep 12 02:00:08.189568 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.189516    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2"} err="failed to get container status \"45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2\": rpc error: code = NotFound desc = could not find container \"45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2\": container with ID starting with 45ed1dda31c669f6dfbe0f7623c3e4d5eb913c4d929e20bbb8730598199605b2 not found: ID does not exist"
Sep 12 02:00:08.205459 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.205417    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"]
Sep 12 02:00:08.207984 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.207961    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-29d2f735-64df6f79-jjn28"]
Sep 12 02:00:08.246960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.246927    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khj4p\" (UniqueName: \"kubernetes.io/projected/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-kube-api-access-khj4p\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:08.246960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.246956    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:08.981562 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.981527    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="490c3b36-80dc-4afc-8fd9-6553aa35f041" path="/var/lib/kubelet/pods/490c3b36-80dc-4afc-8fd9-6553aa35f041/volumes"
Sep 12 02:00:08.982029 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.981848    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0" path="/var/lib/kubelet/pods/7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0/volumes"
Sep 12 02:00:08.982175 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:08.982160    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20" path="/var/lib/kubelet/pods/d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20/volumes"
Sep 12 02:00:09.179510 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.179478    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"]
Sep 12 02:00:09.180118 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.180078    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy" containerID="cri-o://c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb" gracePeriod=30
Sep 12 02:00:09.509769 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.509730    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"]
Sep 12 02:00:09.510275 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510255    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20" containerName="payload-processing"
Sep 12 02:00:09.510275 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510277    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20" containerName="payload-processing"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510290    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="446090eb-e94f-4101-8876-791e5f3e63b9" containerName="cleanup"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510299    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="446090eb-e94f-4101-8876-791e5f3e63b9" containerName="cleanup"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510328    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df3dbb6b-d87f-41e7-826d-298cfae16122" containerName="cleanup"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510338    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="df3dbb6b-d87f-41e7-826d-298cfae16122" containerName="cleanup"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510363    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0" containerName="cleanup"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510370    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0" containerName="cleanup"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510381    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ac7959-254c-4512-9917-1666ea8a04e2" containerName="cleanup"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510388    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ac7959-254c-4512-9917-1666ea8a04e2" containerName="cleanup"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510405    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490c3b36-80dc-4afc-8fd9-6553aa35f041" containerName="revoke-keys"
Sep 12 02:00:09.510443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510418    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="490c3b36-80dc-4afc-8fd9-6553aa35f041" containerName="revoke-keys"
Sep 12 02:00:09.510886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510522    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="71ac7959-254c-4512-9917-1666ea8a04e2" containerName="cleanup"
Sep 12 02:00:09.510886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510534    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="446090eb-e94f-4101-8876-791e5f3e63b9" containerName="cleanup"
Sep 12 02:00:09.510886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510544    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="df3dbb6b-d87f-41e7-826d-298cfae16122" containerName="cleanup"
Sep 12 02:00:09.510886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510553    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0da6dd7-e0d3-4d3a-9a5b-ceb66ac74d20" containerName="payload-processing"
Sep 12 02:00:09.510886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510564    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="490c3b36-80dc-4afc-8fd9-6553aa35f041" containerName="revoke-keys"
Sep 12 02:00:09.510886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.510575    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7379ba0d-4e31-4f54-90dc-ce4d0bbc3dc0" containerName="cleanup"
Sep 12 02:00:09.513915 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.513893    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.516430 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.516407    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-9ef13e\"/\"kube-root-ca.crt\""
Sep 12 02:00:09.516430 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.516407    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-9ef13e\"/\"autores-model-9ef13e-kserve-self-signed-certs\""
Sep 12 02:00:09.516608 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.516422    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-9ef13e\"/\"default-dockercfg-s7wwv\""
Sep 12 02:00:09.516608 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.516558    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-9ef13e\"/\"openshift-service-ca.crt\""
Sep 12 02:00:09.524710 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.524689    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"]
Sep 12 02:00:09.660064 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.660028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tls-certs\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.660064 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.660063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-model-cache\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.660383 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.660236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb5q6\" (UniqueName: \"kubernetes.io/projected/a71999a6-af8b-48c7-9cd8-c96e0668fe78-kube-api-access-cb5q6\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.660383 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.660312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-dshm\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.660383 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.660357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-home\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.660517 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.660383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tmp-dir\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.761625 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.761539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb5q6\" (UniqueName: \"kubernetes.io/projected/a71999a6-af8b-48c7-9cd8-c96e0668fe78-kube-api-access-cb5q6\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.761625 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.761590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-dshm\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.761848 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.761686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-home\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.761848 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.761704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tmp-dir\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.761848 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.761732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tls-certs\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.761848 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.761748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-model-cache\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.762164 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.762140    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-home\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.762263 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.762241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tmp-dir\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.762314 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.762275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-model-cache\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.764096 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.764061    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-dshm\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.764305 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.764289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tls-certs\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.775350 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.775326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb5q6\" (UniqueName: \"kubernetes.io/projected/a71999a6-af8b-48c7-9cd8-c96e0668fe78-kube-api-access-cb5q6\") pod \"autores-model-9ef13e-kserve-6c86fc685f-ddm9d\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") " pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.825530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.825493    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:09.918636 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.918601    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"]
Sep 12 02:00:09.922117 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.922092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:09.924743 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.924720    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e557ff8f-openshift-default-dockercfg-ctxfw\""
Sep 12 02:00:09.937276 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:09.937248    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"]
Sep 12 02:00:10.065403 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.065312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-data\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.065403 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.065361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nftxx\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-kube-api-access-nftxx\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.065403 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.065385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-podinfo\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.065838 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.065436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e5da7a-4501-466e-adcf-952d9864d5c5-istiod-ca-cert\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.065838 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.065517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-credential-socket\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.065838 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.065554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-certs\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.065838 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.065572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-token\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.065838 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.065616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-socket\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.065838 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.065693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-envoy\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.116413 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.116382    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"]
Sep 12 02:00:10.118281 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:10.118251    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda71999a6_af8b_48c7_9cd8_c96e0668fe78.slice/crio-9846cc5f86ee1f38ebe150ed45d027585224a94d9ce5402f1751135affe38fd4 WatchSource:0}: Error finding container 9846cc5f86ee1f38ebe150ed45d027585224a94d9ce5402f1751135affe38fd4: Status 404 returned error can't find the container with id 9846cc5f86ee1f38ebe150ed45d027585224a94d9ce5402f1751135affe38fd4
Sep 12 02:00:10.167232 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.167177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-data\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.167378 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.167261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nftxx\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-kube-api-access-nftxx\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.167700 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.167654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-data\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.167816 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.167790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-podinfo\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.167884 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.167869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e5da7a-4501-466e-adcf-952d9864d5c5-istiod-ca-cert\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.167943 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.167919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-credential-socket\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.167996 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.167957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-certs\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.168045 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.167993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-token\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.168100 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.168045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-socket\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.168155 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.168112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-envoy\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.168737 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.168609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e5da7a-4501-466e-adcf-952d9864d5c5-istiod-ca-cert\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.169122 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.168804    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-credential-socket\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.169122 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.169063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-certs\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.169330 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.169198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-socket\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.170054 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.170023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-podinfo\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.170652 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.170634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-envoy\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.186603 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.186573    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-token\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.186842 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.186814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nftxx\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-kube-api-access-nftxx\") pod \"e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") " pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.208158 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.207817    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d" event={"ID":"a71999a6-af8b-48c7-9cd8-c96e0668fe78","Type":"ContainerStarted","Data":"fdb6747b29a7f810cbd69b3b7f9a1897de6aeee55ff8b53d480a5b1c650c9d2d"}
Sep 12 02:00:10.208158 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.207882    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d" event={"ID":"a71999a6-af8b-48c7-9cd8-c96e0668fe78","Type":"ContainerStarted","Data":"9846cc5f86ee1f38ebe150ed45d027585224a94d9ce5402f1751135affe38fd4"}
Sep 12 02:00:10.236064 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.236028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:10.237530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.237475    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d" podStartSLOduration=1.237458735 podStartE2EDuration="1.237458735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:00:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:00:10.233967993 +0000 UTC m=+915.760869072" watchObservedRunningTime="2026-09-12 02:00:10.237458735 +0000 UTC m=+915.764359851"
Sep 12 02:00:10.521847 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.521817    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"]
Sep 12 02:00:10.522428 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:10.522402    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e5da7a_4501_466e_adcf_952d9864d5c5.slice/crio-2e6f519253742f096e4f1d9c6fbbbd429662847034e8362fd592756f75ad0eb7 WatchSource:0}: Error finding container 2e6f519253742f096e4f1d9c6fbbbd429662847034e8362fd592756f75ad0eb7: Status 404 returned error can't find the container with id 2e6f519253742f096e4f1d9c6fbbbd429662847034e8362fd592756f75ad0eb7
Sep 12 02:00:10.524843 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.524810    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:00:10.524921 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.524880    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:00:10.524921 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:10.524914    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:00:11.216968 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:11.216926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" event={"ID":"a7e5da7a-4501-466e-adcf-952d9864d5c5","Type":"ContainerStarted","Data":"af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c"}
Sep 12 02:00:11.217394 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:11.216992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" event={"ID":"a7e5da7a-4501-466e-adcf-952d9864d5c5","Type":"ContainerStarted","Data":"2e6f519253742f096e4f1d9c6fbbbd429662847034e8362fd592756f75ad0eb7"}
Sep 12 02:00:11.236278 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:11.236251    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:11.241936 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:11.241911    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:11.242296 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:11.242255    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" podStartSLOduration=2.2422399889999998 podStartE2EDuration="2.242239989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:00:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:00:11.241125702 +0000 UTC m=+916.768026784" watchObservedRunningTime="2026-09-12 02:00:11.242239989 +0000 UTC m=+916.769141068"
Sep 12 02:00:12.223386 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:12.223352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:12.224483 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:12.224462    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:14.451511 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.451450    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 02:00:14.510402 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510371    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-socket\") pod \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") "
Sep 12 02:00:14.510587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q67rk\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-kube-api-access-q67rk\") pod \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") "
Sep 12 02:00:14.510587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510468    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-certs\") pod \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") "
Sep 12 02:00:14.510587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-podinfo\") pod \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") "
Sep 12 02:00:14.510587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510531    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-token\") pod \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") "
Sep 12 02:00:14.510587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510550    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istiod-ca-cert\") pod \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") "
Sep 12 02:00:14.510587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510573    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-credential-socket\") pod \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") "
Sep 12 02:00:14.510913 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510616    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-envoy\") pod \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") "
Sep 12 02:00:14.510913 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510668    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-data\") pod \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\" (UID: \"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b\") "
Sep 12 02:00:14.510913 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510691    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" (UID: "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:00:14.510913 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" (UID: "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:00:14.511109 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.510985    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" (UID: "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:00:14.511109 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.511047    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:14.511109 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.511065    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:14.511109 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.511093    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-data" (OuterVolumeSpecName: "istio-data") pod "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" (UID: "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:00:14.511356 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.511161    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" (UID: "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:00:14.512831 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.512791    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" (UID: "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:00:14.512956 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.512879    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" (UID: "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:00:14.512956 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.512894    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-kube-api-access-q67rk" (OuterVolumeSpecName: "kube-api-access-q67rk") pod "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" (UID: "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b"). InnerVolumeSpecName "kube-api-access-q67rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:00:14.512956 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.512905    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-token" (OuterVolumeSpecName: "istio-token") pod "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" (UID: "5315ed96-d7bc-4122-905e-7d7f3bfe9f5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:00:14.611726 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.611691    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:14.611726 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.611723    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q67rk\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-kube-api-access-q67rk\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:14.611987 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.611738    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:14.611987 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.611749    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:14.611987 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.611760    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:14.611987 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.611771    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:14.611987 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:14.611782    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:15.243095 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.243056    2580 generic.go:356] "Generic (PLEG): container finished" podID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerID="c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb" exitCode=0
Sep 12 02:00:15.243271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.243110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" event={"ID":"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b","Type":"ContainerDied","Data":"c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb"}
Sep 12 02:00:15.243271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.243127    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"
Sep 12 02:00:15.243271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.243170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d" event={"ID":"5315ed96-d7bc-4122-905e-7d7f3bfe9f5b","Type":"ContainerDied","Data":"aa70c10a86134311014ec28fde4e53c5a98bee9b600a56c19e4518a76ba5422e"}
Sep 12 02:00:15.243271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.243195    2580 scope.go:117] "RemoveContainer" containerID="c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb"
Sep 12 02:00:15.253198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.253176    2580 scope.go:117] "RemoveContainer" containerID="09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0"
Sep 12 02:00:15.261914 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.261894    2580 scope.go:117] "RemoveContainer" containerID="c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb"
Sep 12 02:00:15.262246 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:00:15.262191    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb\": container with ID starting with c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb not found: ID does not exist" containerID="c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb"
Sep 12 02:00:15.262411 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.262260    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb"} err="failed to get container status \"c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb\": rpc error: code = NotFound desc = could not find container \"c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb\": container with ID starting with c1da61da9188b3a26803c39f86f02dc0deeaaf3ab5ecfcb849886d182e0cf9eb not found: ID does not exist"
Sep 12 02:00:15.262411 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.262287    2580 scope.go:117] "RemoveContainer" containerID="09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0"
Sep 12 02:00:15.262713 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:00:15.262685    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0\": container with ID starting with 09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0 not found: ID does not exist" containerID="09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0"
Sep 12 02:00:15.262813 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.262720    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0"} err="failed to get container status \"09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0\": rpc error: code = NotFound desc = could not find container \"09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0\": container with ID starting with 09a3b7297a664dcc1f54b4233a0370b7332317b532f221249e9bc501cb2934f0 not found: ID does not exist"
Sep 12 02:00:15.265795 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.265772    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"]
Sep 12 02:00:15.271691 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:15.271667    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-29d2f735-openshift-default-69748fd8-sth5d"]
Sep 12 02:00:16.979997 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:16.979961    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" path="/var/lib/kubelet/pods/5315ed96-d7bc-4122-905e-7d7f3bfe9f5b/volumes"
Sep 12 02:00:18.101912 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.101870    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"]
Sep 12 02:00:18.102375 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.102359    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy"
Sep 12 02:00:18.102420 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.102379    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy"
Sep 12 02:00:18.102470 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.102461    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy"
Sep 12 02:00:18.106433 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.106417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:18.109097 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.109074    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-e557ff8f-dockercfg-7w6rq\""
Sep 12 02:00:18.109252 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.109143    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-e557ff8f\""
Sep 12 02:00:18.115967 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.115941    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"]
Sep 12 02:00:18.149510 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.149474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx7hg\" (UniqueName: \"kubernetes.io/projected/3091cef2-00a7-4e22-9658-cb22ae12573b-kube-api-access-lx7hg\") pod \"payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx\" (UID: \"3091cef2-00a7-4e22-9658-cb22ae12573b\") " pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:18.149659 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.149529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3091cef2-00a7-4e22-9658-cb22ae12573b-plugins-config-volume\") pod \"payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx\" (UID: \"3091cef2-00a7-4e22-9658-cb22ae12573b\") " pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:18.250475 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.250440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx7hg\" (UniqueName: \"kubernetes.io/projected/3091cef2-00a7-4e22-9658-cb22ae12573b-kube-api-access-lx7hg\") pod \"payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx\" (UID: \"3091cef2-00a7-4e22-9658-cb22ae12573b\") " pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:18.250634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.250499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3091cef2-00a7-4e22-9658-cb22ae12573b-plugins-config-volume\") pod \"payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx\" (UID: \"3091cef2-00a7-4e22-9658-cb22ae12573b\") " pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:18.251248 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.251199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3091cef2-00a7-4e22-9658-cb22ae12573b-plugins-config-volume\") pod \"payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx\" (UID: \"3091cef2-00a7-4e22-9658-cb22ae12573b\") " pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:18.262695 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.262672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx7hg\" (UniqueName: \"kubernetes.io/projected/3091cef2-00a7-4e22-9658-cb22ae12573b-kube-api-access-lx7hg\") pod \"payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx\" (UID: \"3091cef2-00a7-4e22-9658-cb22ae12573b\") " pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:18.419739 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.419655    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:18.695559 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:18.695528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"]
Sep 12 02:00:18.696620 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:18.696589    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3091cef2_00a7_4e22_9658_cb22ae12573b.slice/crio-4a973322bede6653fcb7dc5c5f028bcee69c1c095f9e70c798052eb965e87e19 WatchSource:0}: Error finding container 4a973322bede6653fcb7dc5c5f028bcee69c1c095f9e70c798052eb965e87e19: Status 404 returned error can't find the container with id 4a973322bede6653fcb7dc5c5f028bcee69c1c095f9e70c798052eb965e87e19
Sep 12 02:00:19.275539 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:19.275481    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx" event={"ID":"3091cef2-00a7-4e22-9658-cb22ae12573b","Type":"ContainerStarted","Data":"8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2"}
Sep 12 02:00:19.275932 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:19.275564    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx" event={"ID":"3091cef2-00a7-4e22-9658-cb22ae12573b","Type":"ContainerStarted","Data":"4a973322bede6653fcb7dc5c5f028bcee69c1c095f9e70c798052eb965e87e19"}
Sep 12 02:00:19.275932 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:19.275613    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:19.294840 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:19.294786    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx" podStartSLOduration=1.294772056 podStartE2EDuration="1.294772056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:00:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:00:19.292090227 +0000 UTC m=+924.818991319" watchObservedRunningTime="2026-09-12 02:00:19.294772056 +0000 UTC m=+924.821673171"
Sep 12 02:00:19.826493 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:19.826448    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:19.826675 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:19.826507    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:19.839587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:19.839556    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:20.293731 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:20.293695    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:00:30.285393 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:30.284367    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:37.398722 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:37.398679    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerID="af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c" exitCode=0
Sep 12 02:00:37.399156 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:37.398757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" event={"ID":"a7e5da7a-4501-466e-adcf-952d9864d5c5","Type":"ContainerDied","Data":"af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c"}
Sep 12 02:00:37.399231 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:37.399171    2580 scope.go:117] "RemoveContainer" containerID="af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c"
Sep 12 02:00:37.400364 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:37.400338    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:00:37.400432 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:37.400408    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:00:37.400478 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:37.400438    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:00:38.407498 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:38.407463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" event={"ID":"a7e5da7a-4501-466e-adcf-952d9864d5c5","Type":"ContainerStarted","Data":"05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a"}
Sep 12 02:00:39.236496 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:39.236457    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:39.237802 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:39.237772    2580 patch_prober.go:28] interesting pod/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 02:00:39.237948 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:39.237831    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 02:00:40.237036 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:40.237001    2580 patch_prober.go:28] interesting pod/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 02:00:40.237500 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:40.237066    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 02:00:41.237362 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:41.237328    2580 patch_prober.go:28] interesting pod/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 02:00:41.237762 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:41.237381    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 02:00:42.223635 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.223598    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:42.237098 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.237069    2580 patch_prober.go:28] interesting pod/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 02:00:42.237236 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.237131    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 02:00:42.575299 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.575256    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"]
Sep 12 02:00:42.575940 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.575914    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy"
Sep 12 02:00:42.575940 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.575942    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy"
Sep 12 02:00:42.576109 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.576072    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5315ed96-d7bc-4122-905e-7d7f3bfe9f5b" containerName="istio-proxy"
Sep 12 02:00:42.580016 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.579994    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"
Sep 12 02:00:42.584302 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.584281    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:00:42.584482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.584458    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:00:42.598229 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.596377    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"]
Sep 12 02:00:42.626667 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.626627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd3f1e5-4f4c-4f97-9380-7b7a921f397c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e557ff8f-sls85\" (UID: \"edd3f1e5-4f4c-4f97-9380-7b7a921f397c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"
Sep 12 02:00:42.727663 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.727624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd3f1e5-4f4c-4f97-9380-7b7a921f397c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e557ff8f-sls85\" (UID: \"edd3f1e5-4f4c-4f97-9380-7b7a921f397c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"
Sep 12 02:00:42.728313 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.728292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd3f1e5-4f4c-4f97-9380-7b7a921f397c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e557ff8f-sls85\" (UID: \"edd3f1e5-4f4c-4f97-9380-7b7a921f397c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"
Sep 12 02:00:42.896368 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:42.896272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"
Sep 12 02:00:43.168753 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:43.168715    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"]
Sep 12 02:00:43.178536 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:43.178501    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd3f1e5_4f4c_4f97_9380_7b7a921f397c.slice/crio-9f7192d0b982872231b942bfbfb20aef6b57040498469bbce5d79d84804b568c WatchSource:0}: Error finding container 9f7192d0b982872231b942bfbfb20aef6b57040498469bbce5d79d84804b568c: Status 404 returned error can't find the container with id 9f7192d0b982872231b942bfbfb20aef6b57040498469bbce5d79d84804b568c
Sep 12 02:00:43.237163 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:43.237137    2580 patch_prober.go:28] interesting pod/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 12 02:00:43.237298 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:43.237199    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 12 02:00:43.441440 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:43.441333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85" event={"ID":"edd3f1e5-4f4c-4f97-9380-7b7a921f397c","Type":"ContainerStarted","Data":"9f7192d0b982872231b942bfbfb20aef6b57040498469bbce5d79d84804b568c"}
Sep 12 02:00:44.241277 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:44.241249    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:44.242417 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:44.242393    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:00:44.453958 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:44.453854    2580 generic.go:356] "Generic (PLEG): container finished" podID="edd3f1e5-4f4c-4f97-9380-7b7a921f397c" containerID="48b11e9318a1cd8a95fbbdbba995eb9e4c56c64ff1218b12535c2d03333d195e" exitCode=0
Sep 12 02:00:44.453958 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:44.453893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85" event={"ID":"edd3f1e5-4f4c-4f97-9380-7b7a921f397c","Type":"ContainerDied","Data":"48b11e9318a1cd8a95fbbdbba995eb9e4c56c64ff1218b12535c2d03333d195e"}
Sep 12 02:00:45.234739 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.234709    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"]
Sep 12 02:00:45.238404 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.238388    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"
Sep 12 02:00:45.245861 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.245825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"]
Sep 12 02:00:45.354577 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.354540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4982dd55-8cf0-4659-988a-402a98d4bd31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9\" (UID: \"4982dd55-8cf0-4659-988a-402a98d4bd31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"
Sep 12 02:00:45.455237 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.455177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4982dd55-8cf0-4659-988a-402a98d4bd31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9\" (UID: \"4982dd55-8cf0-4659-988a-402a98d4bd31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"
Sep 12 02:00:45.455812 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.455791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4982dd55-8cf0-4659-988a-402a98d4bd31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9\" (UID: \"4982dd55-8cf0-4659-988a-402a98d4bd31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"
Sep 12 02:00:45.550778 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.550747    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"
Sep 12 02:00:45.604886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.604852    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"
Sep 12 02:00:45.657054 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.657024    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd3f1e5-4f4c-4f97-9380-7b7a921f397c-maas-api-service-ca\") pod \"edd3f1e5-4f4c-4f97-9380-7b7a921f397c\" (UID: \"edd3f1e5-4f4c-4f97-9380-7b7a921f397c\") "
Sep 12 02:00:45.657416 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.657388    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd3f1e5-4f4c-4f97-9380-7b7a921f397c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "edd3f1e5-4f4c-4f97-9380-7b7a921f397c" (UID: "edd3f1e5-4f4c-4f97-9380-7b7a921f397c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:00:45.758700 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.758663    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd3f1e5-4f4c-4f97-9380-7b7a921f397c-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:45.827423 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:45.827395    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"]
Sep 12 02:00:45.828711 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:00:45.828685    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4982dd55_8cf0_4659_988a_402a98d4bd31.slice/crio-bef05b6dce63423345daa3194c275c296069211f352134691f8f94d7f215f7fd WatchSource:0}: Error finding container bef05b6dce63423345daa3194c275c296069211f352134691f8f94d7f215f7fd: Status 404 returned error can't find the container with id bef05b6dce63423345daa3194c275c296069211f352134691f8f94d7f215f7fd
Sep 12 02:00:46.468936 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:46.468699    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"
Sep 12 02:00:46.468936 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:46.468730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85" event={"ID":"edd3f1e5-4f4c-4f97-9380-7b7a921f397c","Type":"ContainerDied","Data":"9f7192d0b982872231b942bfbfb20aef6b57040498469bbce5d79d84804b568c"}
Sep 12 02:00:46.468936 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:46.468789    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f7192d0b982872231b942bfbfb20aef6b57040498469bbce5d79d84804b568c"
Sep 12 02:00:46.471135 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:46.471108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9" event={"ID":"4982dd55-8cf0-4659-988a-402a98d4bd31","Type":"ContainerStarted","Data":"d85665af8eedcdb4d6315fe9bdaf617d0429c4662f13dfce5db6bdd18e5a8bb7"}
Sep 12 02:00:46.471313 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:46.471159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9" event={"ID":"4982dd55-8cf0-4659-988a-402a98d4bd31","Type":"ContainerStarted","Data":"bef05b6dce63423345daa3194c275c296069211f352134691f8f94d7f215f7fd"}
Sep 12 02:00:46.487934 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:46.487890    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9" podStartSLOduration=1.48787469 podStartE2EDuration="1.48787469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:00:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:00:46.485730764 +0000 UTC m=+952.012631844" watchObservedRunningTime="2026-09-12 02:00:46.48787469 +0000 UTC m=+952.014775769"
Sep 12 02:00:47.481304 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:47.481261    2580 generic.go:356] "Generic (PLEG): container finished" podID="4982dd55-8cf0-4659-988a-402a98d4bd31" containerID="d85665af8eedcdb4d6315fe9bdaf617d0429c4662f13dfce5db6bdd18e5a8bb7" exitCode=0
Sep 12 02:00:47.481692 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:47.481335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9" event={"ID":"4982dd55-8cf0-4659-988a-402a98d4bd31","Type":"ContainerDied","Data":"d85665af8eedcdb4d6315fe9bdaf617d0429c4662f13dfce5db6bdd18e5a8bb7"}
Sep 12 02:00:48.637555 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:48.637506    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"
Sep 12 02:00:48.685560 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:48.685529    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4982dd55-8cf0-4659-988a-402a98d4bd31-maas-api-service-ca\") pod \"4982dd55-8cf0-4659-988a-402a98d4bd31\" (UID: \"4982dd55-8cf0-4659-988a-402a98d4bd31\") "
Sep 12 02:00:48.685888 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:48.685861    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4982dd55-8cf0-4659-988a-402a98d4bd31-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4982dd55-8cf0-4659-988a-402a98d4bd31" (UID: "4982dd55-8cf0-4659-988a-402a98d4bd31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:00:48.786320 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:48.786282    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4982dd55-8cf0-4659-988a-402a98d4bd31-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:49.496394 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:49.496357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9" event={"ID":"4982dd55-8cf0-4659-988a-402a98d4bd31","Type":"ContainerDied","Data":"bef05b6dce63423345daa3194c275c296069211f352134691f8f94d7f215f7fd"}
Sep 12 02:00:49.496394 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:49.496390    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"
Sep 12 02:00:49.496609 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:49.496440    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bef05b6dce63423345daa3194c275c296069211f352134691f8f94d7f215f7fd"
Sep 12 02:00:54.728013 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:54.727976    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"]
Sep 12 02:00:54.728520 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:54.728249    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx" podUID="3091cef2-00a7-4e22-9658-cb22ae12573b" containerName="payload-processing" containerID="cri-o://8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2" gracePeriod=30
Sep 12 02:00:54.879897 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:54.879865    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"]
Sep 12 02:00:54.883361 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:54.883329    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e557ff8f-sls85"]
Sep 12 02:00:54.994613 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:54.994163    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edd3f1e5-4f4c-4f97-9380-7b7a921f397c" path="/var/lib/kubelet/pods/edd3f1e5-4f4c-4f97-9380-7b7a921f397c/volumes"
Sep 12 02:00:55.013806 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.013775    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"]
Sep 12 02:00:55.014403 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.014344    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy" containerID="cri-o://05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a" gracePeriod=30
Sep 12 02:00:55.043176 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.043150    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:55.149286 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.149195    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx7hg\" (UniqueName: \"kubernetes.io/projected/3091cef2-00a7-4e22-9658-cb22ae12573b-kube-api-access-lx7hg\") pod \"3091cef2-00a7-4e22-9658-cb22ae12573b\" (UID: \"3091cef2-00a7-4e22-9658-cb22ae12573b\") "
Sep 12 02:00:55.149455 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.149385    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3091cef2-00a7-4e22-9658-cb22ae12573b-plugins-config-volume\") pod \"3091cef2-00a7-4e22-9658-cb22ae12573b\" (UID: \"3091cef2-00a7-4e22-9658-cb22ae12573b\") "
Sep 12 02:00:55.149772 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.149746    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3091cef2-00a7-4e22-9658-cb22ae12573b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3091cef2-00a7-4e22-9658-cb22ae12573b" (UID: "3091cef2-00a7-4e22-9658-cb22ae12573b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:00:55.151376 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.151349    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3091cef2-00a7-4e22-9658-cb22ae12573b-kube-api-access-lx7hg" (OuterVolumeSpecName: "kube-api-access-lx7hg") pod "3091cef2-00a7-4e22-9658-cb22ae12573b" (UID: "3091cef2-00a7-4e22-9658-cb22ae12573b"). InnerVolumeSpecName "kube-api-access-lx7hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:00:55.250328 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.250239    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3091cef2-00a7-4e22-9658-cb22ae12573b-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:55.250328 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.250262    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx7hg\" (UniqueName: \"kubernetes.io/projected/3091cef2-00a7-4e22-9658-cb22ae12573b-kube-api-access-lx7hg\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:55.535743 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.535642    2580 generic.go:356] "Generic (PLEG): container finished" podID="3091cef2-00a7-4e22-9658-cb22ae12573b" containerID="8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2" exitCode=0
Sep 12 02:00:55.535743 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.535701    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"
Sep 12 02:00:55.535743 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.535734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx" event={"ID":"3091cef2-00a7-4e22-9658-cb22ae12573b","Type":"ContainerDied","Data":"8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2"}
Sep 12 02:00:55.536021 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.535785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx" event={"ID":"3091cef2-00a7-4e22-9658-cb22ae12573b","Type":"ContainerDied","Data":"4a973322bede6653fcb7dc5c5f028bcee69c1c095f9e70c798052eb965e87e19"}
Sep 12 02:00:55.536021 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.535809    2580 scope.go:117] "RemoveContainer" containerID="8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2"
Sep 12 02:00:55.546703 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.546680    2580 scope.go:117] "RemoveContainer" containerID="8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2"
Sep 12 02:00:55.546993 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:00:55.546971    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2\": container with ID starting with 8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2 not found: ID does not exist" containerID="8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2"
Sep 12 02:00:55.547039 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.547002    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2"} err="failed to get container status \"8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2\": rpc error: code = NotFound desc = could not find container \"8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2\": container with ID starting with 8b9cccf8a91bd037dba66976bd6ad049afc00cffd6b7335a1fb092cdb5e3cee2 not found: ID does not exist"
Sep 12 02:00:55.561133 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.561109    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"]
Sep 12 02:00:55.565361 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:55.565329    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e557ff8f-6dc58b7cfc-jmgkx"]
Sep 12 02:00:56.742069 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:56.742032    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7"]
Sep 12 02:00:56.745544 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:56.745517    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-9ef13e-29819640-s8hb7"]
Sep 12 02:00:56.778519 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:56.777552    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"]
Sep 12 02:00:56.778519 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:56.778047    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z" podUID="7be6a912-f702-406c-ab94-bc038b3c5513" containerName="payload-pre-processing" containerID="cri-o://a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621" gracePeriod=30
Sep 12 02:00:56.990175 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:56.990141    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3091cef2-00a7-4e22-9658-cb22ae12573b" path="/var/lib/kubelet/pods/3091cef2-00a7-4e22-9658-cb22ae12573b/volumes"
Sep 12 02:00:56.991609 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:56.991550    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="446090eb-e94f-4101-8876-791e5f3e63b9" path="/var/lib/kubelet/pods/446090eb-e94f-4101-8876-791e5f3e63b9/volumes"
Sep 12 02:00:57.052295 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.052269    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 02:00:57.123111 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.123078    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"]
Sep 12 02:00:57.129157 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.129125    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9ef13e-jhsp9"]
Sep 12 02:00:57.173471 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.173445    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsqqh\" (UniqueName: \"kubernetes.io/projected/7be6a912-f702-406c-ab94-bc038b3c5513-kube-api-access-lsqqh\") pod \"7be6a912-f702-406c-ab94-bc038b3c5513\" (UID: \"7be6a912-f702-406c-ab94-bc038b3c5513\") "
Sep 12 02:00:57.173624 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.173533    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7be6a912-f702-406c-ab94-bc038b3c5513-plugins-config-volume\") pod \"7be6a912-f702-406c-ab94-bc038b3c5513\" (UID: \"7be6a912-f702-406c-ab94-bc038b3c5513\") "
Sep 12 02:00:57.174000 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.173967    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7be6a912-f702-406c-ab94-bc038b3c5513-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7be6a912-f702-406c-ab94-bc038b3c5513" (UID: "7be6a912-f702-406c-ab94-bc038b3c5513"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:00:57.175553 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.175529    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be6a912-f702-406c-ab94-bc038b3c5513-kube-api-access-lsqqh" (OuterVolumeSpecName: "kube-api-access-lsqqh") pod "7be6a912-f702-406c-ab94-bc038b3c5513" (UID: "7be6a912-f702-406c-ab94-bc038b3c5513"). InnerVolumeSpecName "kube-api-access-lsqqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:00:57.275277 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.275238    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsqqh\" (UniqueName: \"kubernetes.io/projected/7be6a912-f702-406c-ab94-bc038b3c5513-kube-api-access-lsqqh\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:57.275277 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.275280    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7be6a912-f702-406c-ab94-bc038b3c5513-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:00:57.551821 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.551785    2580 generic.go:356] "Generic (PLEG): container finished" podID="7be6a912-f702-406c-ab94-bc038b3c5513" containerID="a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621" exitCode=0
Sep 12 02:00:57.551821 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.551825    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"
Sep 12 02:00:57.552112 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.551874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z" event={"ID":"7be6a912-f702-406c-ab94-bc038b3c5513","Type":"ContainerDied","Data":"a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621"}
Sep 12 02:00:57.552112 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.551933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z" event={"ID":"7be6a912-f702-406c-ab94-bc038b3c5513","Type":"ContainerDied","Data":"573f53c6cb1e964ecc3cd0b97f2228acd9e6550df2c89ce917a4141f7bdfd5a6"}
Sep 12 02:00:57.552112 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.551967    2580 scope.go:117] "RemoveContainer" containerID="a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621"
Sep 12 02:00:57.562273 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.562246    2580 scope.go:117] "RemoveContainer" containerID="a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621"
Sep 12 02:00:57.562559 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:00:57.562541    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621\": container with ID starting with a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621 not found: ID does not exist" containerID="a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621"
Sep 12 02:00:57.562630 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.562573    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621"} err="failed to get container status \"a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621\": rpc error: code = NotFound desc = could not find container \"a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621\": container with ID starting with a8fdbce2bd0017ccaf8e4145acbe5c625dd34cce3433d32d579e224f958b3621 not found: ID does not exist"
Sep 12 02:00:57.575491 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.575455    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"]
Sep 12 02:00:57.578158 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:57.578134    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9ef13e-78f47fb5f8-htr5z"]
Sep 12 02:00:58.980309 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:58.980272    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4982dd55-8cf0-4659-988a-402a98d4bd31" path="/var/lib/kubelet/pods/4982dd55-8cf0-4659-988a-402a98d4bd31/volumes"
Sep 12 02:00:58.980690 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:58.980676    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7be6a912-f702-406c-ab94-bc038b3c5513" path="/var/lib/kubelet/pods/7be6a912-f702-406c-ab94-bc038b3c5513/volumes"
Sep 12 02:00:59.244269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:59.244218    2580 patch_prober.go:28] interesting pod/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 12 02:00:59.244432 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:00:59.244290    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 12 02:01:00.280611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.280585    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:01:00.406856 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.406765    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-token\") pod \"a7e5da7a-4501-466e-adcf-952d9864d5c5\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") "
Sep 12 02:01:00.406856 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.406810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e5da7a-4501-466e-adcf-952d9864d5c5-istiod-ca-cert\") pod \"a7e5da7a-4501-466e-adcf-952d9864d5c5\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") "
Sep 12 02:01:00.406856 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.406848    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-certs\") pod \"a7e5da7a-4501-466e-adcf-952d9864d5c5\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") "
Sep 12 02:01:00.407115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.406871    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-socket\") pod \"a7e5da7a-4501-466e-adcf-952d9864d5c5\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") "
Sep 12 02:01:00.407115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.406894    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-podinfo\") pod \"a7e5da7a-4501-466e-adcf-952d9864d5c5\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") "
Sep 12 02:01:00.407115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.406927    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-envoy\") pod \"a7e5da7a-4501-466e-adcf-952d9864d5c5\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") "
Sep 12 02:01:00.407115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.406969    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-data\") pod \"a7e5da7a-4501-466e-adcf-952d9864d5c5\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") "
Sep 12 02:01:00.407115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407022    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nftxx\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-kube-api-access-nftxx\") pod \"a7e5da7a-4501-466e-adcf-952d9864d5c5\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") "
Sep 12 02:01:00.407115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407048    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-credential-socket\") pod \"a7e5da7a-4501-466e-adcf-952d9864d5c5\" (UID: \"a7e5da7a-4501-466e-adcf-952d9864d5c5\") "
Sep 12 02:01:00.407476 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407120    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a7e5da7a-4501-466e-adcf-952d9864d5c5" (UID: "a7e5da7a-4501-466e-adcf-952d9864d5c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:00.407476 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407245    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7e5da7a-4501-466e-adcf-952d9864d5c5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a7e5da7a-4501-466e-adcf-952d9864d5c5" (UID: "a7e5da7a-4501-466e-adcf-952d9864d5c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:01:00.407476 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407261    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a7e5da7a-4501-466e-adcf-952d9864d5c5" (UID: "a7e5da7a-4501-466e-adcf-952d9864d5c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:00.407476 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407407    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-data" (OuterVolumeSpecName: "istio-data") pod "a7e5da7a-4501-466e-adcf-952d9864d5c5" (UID: "a7e5da7a-4501-466e-adcf-952d9864d5c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:00.407661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407472    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a7e5da7a-4501-466e-adcf-952d9864d5c5" (UID: "a7e5da7a-4501-466e-adcf-952d9864d5c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:00.407661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407556    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:00.407661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407572    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:00.407661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407586    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e5da7a-4501-466e-adcf-952d9864d5c5-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:00.407661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407598    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:00.407661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.407610    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:00.409281 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.409261    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "a7e5da7a-4501-466e-adcf-952d9864d5c5" (UID: "a7e5da7a-4501-466e-adcf-952d9864d5c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:01:00.409281 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.409268    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-kube-api-access-nftxx" (OuterVolumeSpecName: "kube-api-access-nftxx") pod "a7e5da7a-4501-466e-adcf-952d9864d5c5" (UID: "a7e5da7a-4501-466e-adcf-952d9864d5c5"). InnerVolumeSpecName "kube-api-access-nftxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:01:00.409567 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.409546    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a7e5da7a-4501-466e-adcf-952d9864d5c5" (UID: "a7e5da7a-4501-466e-adcf-952d9864d5c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:00.409631 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.409551    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a7e5da7a-4501-466e-adcf-952d9864d5c5" (UID: "a7e5da7a-4501-466e-adcf-952d9864d5c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:01:00.508554 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.508513    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nftxx\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-kube-api-access-nftxx\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:00.508554 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.508547    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:00.508554 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.508557    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:00.508850 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.508567    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e5da7a-4501-466e-adcf-952d9864d5c5-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:00.580095 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.580051    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerID="05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a" exitCode=0
Sep 12 02:01:00.580095 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.580089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" event={"ID":"a7e5da7a-4501-466e-adcf-952d9864d5c5","Type":"ContainerDied","Data":"05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a"}
Sep 12 02:01:00.580306 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.580110    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"
Sep 12 02:01:00.580306 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.580142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv" event={"ID":"a7e5da7a-4501-466e-adcf-952d9864d5c5","Type":"ContainerDied","Data":"2e6f519253742f096e4f1d9c6fbbbd429662847034e8362fd592756f75ad0eb7"}
Sep 12 02:01:00.580306 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.580167    2580 scope.go:117] "RemoveContainer" containerID="05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a"
Sep 12 02:01:00.590700 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.590680    2580 scope.go:117] "RemoveContainer" containerID="af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c"
Sep 12 02:01:00.599922 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.599898    2580 scope.go:117] "RemoveContainer" containerID="05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a"
Sep 12 02:01:00.600590 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:01:00.600568    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a\": container with ID starting with 05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a not found: ID does not exist" containerID="05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a"
Sep 12 02:01:00.600677 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.600597    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a"} err="failed to get container status \"05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a\": rpc error: code = NotFound desc = could not find container \"05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a\": container with ID starting with 05b0320e529008ed3a27203960a3a88dae5afff4dd577849050113bac87c964a not found: ID does not exist"
Sep 12 02:01:00.600677 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.600615    2580 scope.go:117] "RemoveContainer" containerID="af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c"
Sep 12 02:01:00.600954 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:01:00.600920    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c\": container with ID starting with af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c not found: ID does not exist" containerID="af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c"
Sep 12 02:01:00.600954 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.600941    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c"} err="failed to get container status \"af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c\": rpc error: code = NotFound desc = could not find container \"af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c\": container with ID starting with af20e2e779a92fb9b0160c212a9c11cbd121e4dcd7aeaaeb9b5d1d8bf621c85c not found: ID does not exist"
Sep 12 02:01:00.602697 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.602676    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"]
Sep 12 02:01:00.608756 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.608726    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e557ff8f-openshift-default-64949975fb-g5jnv"]
Sep 12 02:01:00.980928 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:00.980886    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" path="/var/lib/kubelet/pods/a7e5da7a-4501-466e-adcf-952d9864d5c5/volumes"
Sep 12 02:01:02.485038 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:02.484995    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"]
Sep 12 02:01:17.504689 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.504602    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d" podUID="a71999a6-af8b-48c7-9cd8-c96e0668fe78" containerName="main" containerID="cri-o://fdb6747b29a7f810cbd69b3b7f9a1897de6aeee55ff8b53d480a5b1c650c9d2d" gracePeriod=45
Sep 12 02:01:17.693693 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.693658    2580 generic.go:356] "Generic (PLEG): container finished" podID="a71999a6-af8b-48c7-9cd8-c96e0668fe78" containerID="fdb6747b29a7f810cbd69b3b7f9a1897de6aeee55ff8b53d480a5b1c650c9d2d" exitCode=0
Sep 12 02:01:17.693845 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.693719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d" event={"ID":"a71999a6-af8b-48c7-9cd8-c96e0668fe78","Type":"ContainerDied","Data":"fdb6747b29a7f810cbd69b3b7f9a1897de6aeee55ff8b53d480a5b1c650c9d2d"}
Sep 12 02:01:17.763063 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.762999    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:01:17.877325 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877292    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-home\") pod \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") "
Sep 12 02:01:17.877530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877350    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-dshm\") pod \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") "
Sep 12 02:01:17.877530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877378    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb5q6\" (UniqueName: \"kubernetes.io/projected/a71999a6-af8b-48c7-9cd8-c96e0668fe78-kube-api-access-cb5q6\") pod \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") "
Sep 12 02:01:17.877530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877405    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-model-cache\") pod \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") "
Sep 12 02:01:17.877530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877512    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tmp-dir\") pod \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") "
Sep 12 02:01:17.877770 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877544    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tls-certs\") pod \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\" (UID: \"a71999a6-af8b-48c7-9cd8-c96e0668fe78\") "
Sep 12 02:01:17.877770 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877614    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-home" (OuterVolumeSpecName: "home") pod "a71999a6-af8b-48c7-9cd8-c96e0668fe78" (UID: "a71999a6-af8b-48c7-9cd8-c96e0668fe78"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:17.877874 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877765    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-model-cache" (OuterVolumeSpecName: "model-cache") pod "a71999a6-af8b-48c7-9cd8-c96e0668fe78" (UID: "a71999a6-af8b-48c7-9cd8-c96e0668fe78"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:17.877874 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877780    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a71999a6-af8b-48c7-9cd8-c96e0668fe78" (UID: "a71999a6-af8b-48c7-9cd8-c96e0668fe78"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:17.877946 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877911    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:17.877946 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877929    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:17.877946 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.877941    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:17.879870 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.879841    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a71999a6-af8b-48c7-9cd8-c96e0668fe78" (UID: "a71999a6-af8b-48c7-9cd8-c96e0668fe78"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:01:17.879870 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.879851    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a71999a6-af8b-48c7-9cd8-c96e0668fe78-kube-api-access-cb5q6" (OuterVolumeSpecName: "kube-api-access-cb5q6") pod "a71999a6-af8b-48c7-9cd8-c96e0668fe78" (UID: "a71999a6-af8b-48c7-9cd8-c96e0668fe78"). InnerVolumeSpecName "kube-api-access-cb5q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:01:17.880038 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.879933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-dshm" (OuterVolumeSpecName: "dshm") pod "a71999a6-af8b-48c7-9cd8-c96e0668fe78" (UID: "a71999a6-af8b-48c7-9cd8-c96e0668fe78"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:17.978828 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.978793    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71999a6-af8b-48c7-9cd8-c96e0668fe78-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:17.978828 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.978821    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71999a6-af8b-48c7-9cd8-c96e0668fe78-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:17.978828 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:17.978831    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb5q6\" (UniqueName: \"kubernetes.io/projected/a71999a6-af8b-48c7-9cd8-c96e0668fe78-kube-api-access-cb5q6\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:18.702044 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:18.702015    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"
Sep 12 02:01:18.702500 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:18.702050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d" event={"ID":"a71999a6-af8b-48c7-9cd8-c96e0668fe78","Type":"ContainerDied","Data":"9846cc5f86ee1f38ebe150ed45d027585224a94d9ce5402f1751135affe38fd4"}
Sep 12 02:01:18.702500 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:18.702105    2580 scope.go:117] "RemoveContainer" containerID="fdb6747b29a7f810cbd69b3b7f9a1897de6aeee55ff8b53d480a5b1c650c9d2d"
Sep 12 02:01:18.723411 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:18.723384    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"]
Sep 12 02:01:18.727747 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:18.727722    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-9ef13e/autores-model-9ef13e-kserve-6c86fc685f-ddm9d"]
Sep 12 02:01:18.980806 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:18.980703    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a71999a6-af8b-48c7-9cd8-c96e0668fe78" path="/var/lib/kubelet/pods/a71999a6-af8b-48c7-9cd8-c96e0668fe78/volumes"
Sep 12 02:01:27.232844 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.232812    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"]
Sep 12 02:01:27.233401 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233380    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4982dd55-8cf0-4659-988a-402a98d4bd31" containerName="revoke-keys"
Sep 12 02:01:27.233401 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233401    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4982dd55-8cf0-4659-988a-402a98d4bd31" containerName="revoke-keys"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233413    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be6a912-f702-406c-ab94-bc038b3c5513" containerName="payload-pre-processing"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233419    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be6a912-f702-406c-ab94-bc038b3c5513" containerName="payload-pre-processing"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233427    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a71999a6-af8b-48c7-9cd8-c96e0668fe78" containerName="main"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233433    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a71999a6-af8b-48c7-9cd8-c96e0668fe78" containerName="main"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233442    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3091cef2-00a7-4e22-9658-cb22ae12573b" containerName="payload-processing"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233448    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3091cef2-00a7-4e22-9658-cb22ae12573b" containerName="payload-processing"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233459    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233464    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233474    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edd3f1e5-4f4c-4f97-9380-7b7a921f397c" containerName="revoke-keys"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233479    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="edd3f1e5-4f4c-4f97-9380-7b7a921f397c" containerName="revoke-keys"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233500    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233508    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233574    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233581    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3091cef2-00a7-4e22-9658-cb22ae12573b" containerName="payload-processing"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233587    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4982dd55-8cf0-4659-988a-402a98d4bd31" containerName="revoke-keys"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233594    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be6a912-f702-406c-ab94-bc038b3c5513" containerName="payload-pre-processing"
Sep 12 02:01:27.233594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233604    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a71999a6-af8b-48c7-9cd8-c96e0668fe78" containerName="main"
Sep 12 02:01:27.234521 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.233609    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="edd3f1e5-4f4c-4f97-9380-7b7a921f397c" containerName="revoke-keys"
Sep 12 02:01:27.236750 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.236731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.240466 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.240442    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f660a6\"/\"premium-w0-f660a6-kserve-self-signed-certs\""
Sep 12 02:01:27.240659 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.240466    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f660a6\"/\"kube-root-ca.crt\""
Sep 12 02:01:27.240659 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.240442    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f660a6\"/\"openshift-service-ca.crt\""
Sep 12 02:01:27.240659 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.240458    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f660a6\"/\"default-dockercfg-sh8kw\""
Sep 12 02:01:27.244766 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.244724    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"]
Sep 12 02:01:27.263649 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.263622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8dad09b-8fca-4f6c-8339-d81618662fc3-tls-certs\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.263781 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.263675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-home\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.263781 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.263762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4p2x\" (UniqueName: \"kubernetes.io/projected/f8dad09b-8fca-4f6c-8339-d81618662fc3-kube-api-access-v4p2x\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.263908 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.263801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-dshm\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.263908 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.263890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-model-cache\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.264015 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.263923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-tmp-dir\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.364530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.364495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-model-cache\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.364530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.364535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-tmp-dir\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.364791 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.364560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8dad09b-8fca-4f6c-8339-d81618662fc3-tls-certs\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.364791 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.364602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-home\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.364791 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.364641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4p2x\" (UniqueName: \"kubernetes.io/projected/f8dad09b-8fca-4f6c-8339-d81618662fc3-kube-api-access-v4p2x\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.364791 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.364658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-dshm\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.365012 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.364940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-model-cache\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.365012 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.364997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-tmp-dir\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.365115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.365046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-home\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.367011 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.366986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-dshm\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.367198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.367177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8dad09b-8fca-4f6c-8339-d81618662fc3-tls-certs\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.372598 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.372574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4p2x\" (UniqueName: \"kubernetes.io/projected/f8dad09b-8fca-4f6c-8339-d81618662fc3-kube-api-access-v4p2x\") pod \"premium-w0-f660a6-kserve-58f9d867bc-27bnn\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") " pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.547618 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.547583    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:27.825181 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:27.825153    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"]
Sep 12 02:01:27.826623 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:01:27.826588    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8dad09b_8fca_4f6c_8339_d81618662fc3.slice/crio-a62d9929bde1e880f5f3b4a5892d52a889a6156785b995d3421a632d6a8c7a19 WatchSource:0}: Error finding container a62d9929bde1e880f5f3b4a5892d52a889a6156785b995d3421a632d6a8c7a19: Status 404 returned error can't find the container with id a62d9929bde1e880f5f3b4a5892d52a889a6156785b995d3421a632d6a8c7a19
Sep 12 02:01:28.766800 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:28.766754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn" event={"ID":"f8dad09b-8fca-4f6c-8339-d81618662fc3","Type":"ContainerStarted","Data":"3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba"}
Sep 12 02:01:28.767420 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:28.766810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn" event={"ID":"f8dad09b-8fca-4f6c-8339-d81618662fc3","Type":"ContainerStarted","Data":"a62d9929bde1e880f5f3b4a5892d52a889a6156785b995d3421a632d6a8c7a19"}
Sep 12 02:01:28.783875 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:28.783827    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn" podStartSLOduration=1.7838106470000001 podStartE2EDuration="1.783810647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:01:28.782666385 +0000 UTC m=+994.309567466" watchObservedRunningTime="2026-09-12 02:01:28.783810647 +0000 UTC m=+994.310711726"
Sep 12 02:01:35.514423 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.514386    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"]
Sep 12 02:01:35.515078 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.515058    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7e5da7a-4501-466e-adcf-952d9864d5c5" containerName="istio-proxy"
Sep 12 02:01:35.518369 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.518346    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.521164 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.521140    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-98f68e\"/\"premium-w4-98f68e-kserve-self-signed-certs\""
Sep 12 02:01:35.527062 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.527034    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"]
Sep 12 02:01:35.646965 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.646927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fjr9\" (UniqueName: \"kubernetes.io/projected/94dfebbb-6c03-4f93-be2b-2a01ff053c22-kube-api-access-8fjr9\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.646965 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.646964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tls-certs\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.647189 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.647093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-home\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.647189 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.647153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-model-cache\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.647189 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.647182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tmp-dir\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.647339 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.647252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-dshm\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.748008 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.747965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-home\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.748243 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.748039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-model-cache\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.748243 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.748072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tmp-dir\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.748243 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.748121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-dshm\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.748243 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.748175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fjr9\" (UniqueName: \"kubernetes.io/projected/94dfebbb-6c03-4f93-be2b-2a01ff053c22-kube-api-access-8fjr9\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.748243 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.748197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tls-certs\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.748557 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.748445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-home\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.748557 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.748474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tmp-dir\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.748557 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.748513    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-model-cache\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.750470 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.750447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-dshm\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.750759 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.750738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tls-certs\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.756857 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.756839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fjr9\" (UniqueName: \"kubernetes.io/projected/94dfebbb-6c03-4f93-be2b-2a01ff053c22-kube-api-access-8fjr9\") pod \"premium-w4-98f68e-kserve-64859f95d6-2f7rw\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") " pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:35.830271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:35.830160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:36.107084 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:36.107060    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"]
Sep 12 02:01:36.109564 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:01:36.109533    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94dfebbb_6c03_4f93_be2b_2a01ff053c22.slice/crio-b102ba52c9bb2a7a39f29fb8ff2423f54919b7da7244b44d5f80d340f3ea61e8 WatchSource:0}: Error finding container b102ba52c9bb2a7a39f29fb8ff2423f54919b7da7244b44d5f80d340f3ea61e8: Status 404 returned error can't find the container with id b102ba52c9bb2a7a39f29fb8ff2423f54919b7da7244b44d5f80d340f3ea61e8
Sep 12 02:01:36.822938 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:36.822901    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw" event={"ID":"94dfebbb-6c03-4f93-be2b-2a01ff053c22","Type":"ContainerStarted","Data":"12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58"}
Sep 12 02:01:36.823342 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:36.822948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw" event={"ID":"94dfebbb-6c03-4f93-be2b-2a01ff053c22","Type":"ContainerStarted","Data":"b102ba52c9bb2a7a39f29fb8ff2423f54919b7da7244b44d5f80d340f3ea61e8"}
Sep 12 02:01:36.841509 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:36.841453    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw" podStartSLOduration=1.841437379 podStartE2EDuration="1.841437379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:01:36.839443153 +0000 UTC m=+1002.366344244" watchObservedRunningTime="2026-09-12 02:01:36.841437379 +0000 UTC m=+1002.368338459"
Sep 12 02:01:37.547894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:37.547853    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:37.548085 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:37.547908    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:37.560319 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:37.560292    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:37.840164 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:37.840076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:01:39.705399 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:39.705364    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"]
Sep 12 02:01:39.705950 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:39.705596    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy" containerID="cri-o://5435ac5b0cf947b954ffb632311d77daebe597d845901b263c65e4e497d99332" gracePeriod=30
Sep 12 02:01:40.505824 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.505788    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"]
Sep 12 02:01:40.509590 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.509566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.513147 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.513113    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-b838b7-openshift-default-dockercfg-jktbj\""
Sep 12 02:01:40.521884 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.521859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"]
Sep 12 02:01:40.598324 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.598285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-podinfo\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.598324 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.598327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882c1f21-e304-4ca3-8e6c-bbac21cede11-istiod-ca-cert\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.598609 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.598351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-data\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.598609 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.598473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-credential-socket\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.598609 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.598498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-socket\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.598609 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.598548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-certs\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.598609 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.598568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-token\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.598899 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.598654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-envoy\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.598899 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.598684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn6pw\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-kube-api-access-fn6pw\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.699650 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.699609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-credential-socket\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.699650 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.699650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-socket\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.699893 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.699693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-certs\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.699893 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.699713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-token\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.699893 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.699771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-envoy\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.699893 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.699801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn6pw\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-kube-api-access-fn6pw\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.699893 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.699841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-podinfo\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.699893 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.699868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882c1f21-e304-4ca3-8e6c-bbac21cede11-istiod-ca-cert\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.699893 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.699895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-data\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.700353 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.700064    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-credential-socket\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.700353 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.700172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-socket\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.700353 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.700286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-data\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.700528 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.700428    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-certs\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.700833 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.700809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882c1f21-e304-4ca3-8e6c-bbac21cede11-istiod-ca-cert\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.702192 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.702161    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-envoy\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.702441 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.702419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-podinfo\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.708116 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.708077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-token\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.708433 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.708288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn6pw\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-kube-api-access-fn6pw\") pod \"e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") " pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:40.822048 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:40.821963    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:41.107063 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:41.107037    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"]
Sep 12 02:01:41.108270 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:01:41.108238    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882c1f21_e304_4ca3_8e6c_bbac21cede11.slice/crio-da0d6d63de1190c85323692d75b7b4eb3933f8a30cffea90f0d2037949ff6fa8 WatchSource:0}: Error finding container da0d6d63de1190c85323692d75b7b4eb3933f8a30cffea90f0d2037949ff6fa8: Status 404 returned error can't find the container with id da0d6d63de1190c85323692d75b7b4eb3933f8a30cffea90f0d2037949ff6fa8
Sep 12 02:01:41.110629 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:41.110596    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:01:41.110721 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:41.110664    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:01:41.110721 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:41.110693    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:01:41.859205 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:41.859156    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" event={"ID":"882c1f21-e304-4ca3-8e6c-bbac21cede11","Type":"ContainerStarted","Data":"30688fc294c6c445be2d43fc2bd83d2a9bcd47521716d0b37ac1cd413a8f5e2f"}
Sep 12 02:01:41.859669 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:41.859268    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" event={"ID":"882c1f21-e304-4ca3-8e6c-bbac21cede11","Type":"ContainerStarted","Data":"da0d6d63de1190c85323692d75b7b4eb3933f8a30cffea90f0d2037949ff6fa8"}
Sep 12 02:01:41.881181 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:41.881118    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" podStartSLOduration=1.8811007119999998 podStartE2EDuration="1.881100712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:01:41.879610267 +0000 UTC m=+1007.406511348" watchObservedRunningTime="2026-09-12 02:01:41.881100712 +0000 UTC m=+1007.408001791"
Sep 12 02:01:42.822252 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:42.822183    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:42.827511 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:42.827484    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:42.865620 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:42.865591    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:42.866379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:42.866357    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:01:44.885550 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:44.885514    2580 generic.go:356] "Generic (PLEG): container finished" podID="8f8252df-28cf-4c63-be91-91bab798d297" containerID="5435ac5b0cf947b954ffb632311d77daebe597d845901b263c65e4e497d99332" exitCode=0
Sep 12 02:01:44.885946 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:44.885589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" event={"ID":"8f8252df-28cf-4c63-be91-91bab798d297","Type":"ContainerDied","Data":"5435ac5b0cf947b954ffb632311d77daebe597d845901b263c65e4e497d99332"}
Sep 12 02:01:44.885946 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:44.885677    2580 scope.go:117] "RemoveContainer" containerID="6857e9ec9d826bd3009835995e89b1d70ca85709990a70d240d93806fbd63eeb"
Sep 12 02:01:44.979818 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:44.979749    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 02:01:45.148115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148076    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-credential-socket\") pod \"8f8252df-28cf-4c63-be91-91bab798d297\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") "
Sep 12 02:01:45.148344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148126    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-envoy\") pod \"8f8252df-28cf-4c63-be91-91bab798d297\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") "
Sep 12 02:01:45.148344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148155    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-socket\") pod \"8f8252df-28cf-4c63-be91-91bab798d297\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") "
Sep 12 02:01:45.148344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148195    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-certs\") pod \"8f8252df-28cf-4c63-be91-91bab798d297\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") "
Sep 12 02:01:45.148344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148276    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8252df-28cf-4c63-be91-91bab798d297-istiod-ca-cert\") pod \"8f8252df-28cf-4c63-be91-91bab798d297\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") "
Sep 12 02:01:45.148344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148323    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-istio-token\") pod \"8f8252df-28cf-4c63-be91-91bab798d297\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") "
Sep 12 02:01:45.148634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148385    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl5p7\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-kube-api-access-sl5p7\") pod \"8f8252df-28cf-4c63-be91-91bab798d297\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") "
Sep 12 02:01:45.148634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148399    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f8252df-28cf-4c63-be91-91bab798d297" (UID: "8f8252df-28cf-4c63-be91-91bab798d297"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:45.148634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148410    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8252df-28cf-4c63-be91-91bab798d297-istio-podinfo\") pod \"8f8252df-28cf-4c63-be91-91bab798d297\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") "
Sep 12 02:01:45.148634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148458    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-data\") pod \"8f8252df-28cf-4c63-be91-91bab798d297\" (UID: \"8f8252df-28cf-4c63-be91-91bab798d297\") "
Sep 12 02:01:45.148634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148488    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f8252df-28cf-4c63-be91-91bab798d297" (UID: "8f8252df-28cf-4c63-be91-91bab798d297"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:45.148634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148551    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f8252df-28cf-4c63-be91-91bab798d297" (UID: "8f8252df-28cf-4c63-be91-91bab798d297"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:45.148960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f8252df-28cf-4c63-be91-91bab798d297-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f8252df-28cf-4c63-be91-91bab798d297" (UID: "8f8252df-28cf-4c63-be91-91bab798d297"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:01:45.148960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148751    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f8252df-28cf-4c63-be91-91bab798d297" (UID: "8f8252df-28cf-4c63-be91-91bab798d297"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:45.148960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148843    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:45.148960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148861    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:45.148960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148873    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:45.148960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148886    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8252df-28cf-4c63-be91-91bab798d297-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:45.148960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.148898    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:45.150540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.150512    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f8252df-28cf-4c63-be91-91bab798d297" (UID: "8f8252df-28cf-4c63-be91-91bab798d297"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:01:45.150657 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.150643    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f8252df-28cf-4c63-be91-91bab798d297-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f8252df-28cf-4c63-be91-91bab798d297" (UID: "8f8252df-28cf-4c63-be91-91bab798d297"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:01:45.150911 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.150892    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f8252df-28cf-4c63-be91-91bab798d297" (UID: "8f8252df-28cf-4c63-be91-91bab798d297"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:01:45.150953 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.150905    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-kube-api-access-sl5p7" (OuterVolumeSpecName: "kube-api-access-sl5p7") pod "8f8252df-28cf-4c63-be91-91bab798d297" (UID: "8f8252df-28cf-4c63-be91-91bab798d297"). InnerVolumeSpecName "kube-api-access-sl5p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:01:45.249491 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.249458    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8252df-28cf-4c63-be91-91bab798d297-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:45.249491 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.249485    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:45.249491 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.249494    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl5p7\" (UniqueName: \"kubernetes.io/projected/8f8252df-28cf-4c63-be91-91bab798d297-kube-api-access-sl5p7\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:45.249741 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.249511    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8252df-28cf-4c63-be91-91bab798d297-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:01:45.830671 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.830631    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:45.830671 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.830681    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:45.843324 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.843291    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:45.894188 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.894162    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"
Sep 12 02:01:45.894648 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.894205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7" event={"ID":"8f8252df-28cf-4c63-be91-91bab798d297","Type":"ContainerDied","Data":"cc98e32f614b5723c3f388d6d771e0a4ea13e9d5f80b95345b85944895279192"}
Sep 12 02:01:45.894648 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.894285    2580 scope.go:117] "RemoveContainer" containerID="5435ac5b0cf947b954ffb632311d77daebe597d845901b263c65e4e497d99332"
Sep 12 02:01:45.907656 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.907635    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:01:45.918331 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.918303    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"]
Sep 12 02:01:45.922389 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:45.922359    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-9ef13e-openshift-default-79f6785c96-g2bp7"]
Sep 12 02:01:46.980586 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:46.980546    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f8252df-28cf-4c63-be91-91bab798d297" path="/var/lib/kubelet/pods/8f8252df-28cf-4c63-be91-91bab798d297/volumes"
Sep 12 02:01:47.560509 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.560477    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"]
Sep 12 02:01:47.560871 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.560858    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy"
Sep 12 02:01:47.560916 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.560873    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy"
Sep 12 02:01:47.560962 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.560952    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy"
Sep 12 02:01:47.560998 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.560964    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy"
Sep 12 02:01:47.566002 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.565983    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:01:47.568656 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.568632    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-b838b7\""
Sep 12 02:01:47.570819 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.570546    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-b838b7-dockercfg-92dkj\""
Sep 12 02:01:47.574231 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.574189    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"]
Sep 12 02:01:47.676034 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.675995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-plugins-config-volume\") pod \"payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d\" (UID: \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\") " pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:01:47.676198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.676046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmxb8\" (UniqueName: \"kubernetes.io/projected/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-kube-api-access-wmxb8\") pod \"payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d\" (UID: \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\") " pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:01:47.777770 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.777729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-plugins-config-volume\") pod \"payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d\" (UID: \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\") " pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:01:47.777972 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.777788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmxb8\" (UniqueName: \"kubernetes.io/projected/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-kube-api-access-wmxb8\") pod \"payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d\" (UID: \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\") " pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:01:47.778542 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.778516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-plugins-config-volume\") pod \"payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d\" (UID: \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\") " pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:01:47.785678 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.785659    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmxb8\" (UniqueName: \"kubernetes.io/projected/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-kube-api-access-wmxb8\") pod \"payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d\" (UID: \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\") " pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:01:47.880455 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:47.880372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:01:48.155850 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:48.155825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"]
Sep 12 02:01:48.157770 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:01:48.157734    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1243ae_58ad_419e_a1ff_320d9e9ccf37.slice/crio-a536aa9aa96439b73493cfd5c89ea4a3d8b38d7dbae57671f3e41c5db644d650 WatchSource:0}: Error finding container a536aa9aa96439b73493cfd5c89ea4a3d8b38d7dbae57671f3e41c5db644d650: Status 404 returned error can't find the container with id a536aa9aa96439b73493cfd5c89ea4a3d8b38d7dbae57671f3e41c5db644d650
Sep 12 02:01:48.923861 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:48.923826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d" event={"ID":"fd1243ae-58ad-419e-a1ff-320d9e9ccf37","Type":"ContainerStarted","Data":"7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd"}
Sep 12 02:01:48.924058 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:48.923884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d" event={"ID":"fd1243ae-58ad-419e-a1ff-320d9e9ccf37","Type":"ContainerStarted","Data":"a536aa9aa96439b73493cfd5c89ea4a3d8b38d7dbae57671f3e41c5db644d650"}
Sep 12 02:01:48.924058 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:48.923927    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:01:48.940902 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:48.940852    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d" podStartSLOduration=1.940835221 podStartE2EDuration="1.940835221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:01:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:01:48.938936471 +0000 UTC m=+1014.465837552" watchObservedRunningTime="2026-09-12 02:01:48.940835221 +0000 UTC m=+1014.467736323"
Sep 12 02:01:59.932931 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:01:59.932895    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:02:07.058581 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:07.058545    2580 generic.go:356] "Generic (PLEG): container finished" podID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerID="30688fc294c6c445be2d43fc2bd83d2a9bcd47521716d0b37ac1cd413a8f5e2f" exitCode=0
Sep 12 02:02:07.059118 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:07.058622    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" event={"ID":"882c1f21-e304-4ca3-8e6c-bbac21cede11","Type":"ContainerDied","Data":"30688fc294c6c445be2d43fc2bd83d2a9bcd47521716d0b37ac1cd413a8f5e2f"}
Sep 12 02:02:07.059205 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:07.059190    2580 scope.go:117] "RemoveContainer" containerID="30688fc294c6c445be2d43fc2bd83d2a9bcd47521716d0b37ac1cd413a8f5e2f"
Sep 12 02:02:07.060517 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:07.060493    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:02:07.060586 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:07.060564    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:02:07.060624 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:07.060596    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:02:08.068179 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:08.068137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" event={"ID":"882c1f21-e304-4ca3-8e6c-bbac21cede11","Type":"ContainerStarted","Data":"b755e45cca2c8cf702fe3764747f88045d64f05771895e4d070ffe9f2cfa46f0"}
Sep 12 02:02:08.822397 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:08.822356    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:02:08.823766 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:08.823739    2580 patch_prober.go:28] interesting pod/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 12 02:02:08.823884 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:08.823824    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 12 02:02:09.822658 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:09.822619    2580 patch_prober.go:28] interesting pod/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 12 02:02:09.823072 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:09.822674    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 12 02:02:10.822581 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:10.822541    2580 patch_prober.go:28] interesting pod/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 12 02:02:10.822853 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:10.822612    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 12 02:02:11.823082 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:11.823045    2580 patch_prober.go:28] interesting pod/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 12 02:02:11.823504 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:11.823100    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 12 02:02:12.823315 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:12.823277    2580 patch_prober.go:28] interesting pod/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 12 02:02:12.823715 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:12.823337    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 12 02:02:12.866411 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:12.866377    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:02:13.826769 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:13.826737    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:02:13.827744 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:13.827722    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:02:51.618043 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.617968    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"]
Sep 12 02:02:51.618690 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.618667    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy"
Sep 12 02:02:51.618774 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.618695    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f8252df-28cf-4c63-be91-91bab798d297" containerName="istio-proxy"
Sep 12 02:02:51.621111 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.621090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.624958 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.624925    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-b838b7\"/\"kube-root-ca.crt\""
Sep 12 02:02:51.625080 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.625025    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-b838b7\"/\"openshift-service-ca.crt\""
Sep 12 02:02:51.625180 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.624935    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-b838b7\"/\"default-dockercfg-6tphr\""
Sep 12 02:02:51.625313 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.624931    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-b838b7\"/\"explicit-model-b838b7-kserve-self-signed-certs\""
Sep 12 02:02:51.632889 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.632855    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"]
Sep 12 02:02:51.732032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.731998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tmp-dir\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.732246 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.732059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-model-cache\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.732246 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.732105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tls-certs\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.732246 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.732153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtf57\" (UniqueName: \"kubernetes.io/projected/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-kube-api-access-jtf57\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.732246 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.732190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-home\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.732424 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.732300    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-dshm\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.833133 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.833092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-dshm\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.833344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.833149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tmp-dir\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.833344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.833181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-model-cache\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.833344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.833233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tls-certs\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.833344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.833282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtf57\" (UniqueName: \"kubernetes.io/projected/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-kube-api-access-jtf57\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.833344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.833328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-home\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.833698 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.833677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-model-cache\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.833768 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.833702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tmp-dir\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.833768 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.833734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-home\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.835489 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.835469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-dshm\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.835812 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.835795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tls-certs\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.841611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.841584    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtf57\" (UniqueName: \"kubernetes.io/projected/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-kube-api-access-jtf57\") pod \"explicit-model-b838b7-kserve-65db7c769c-nljpx\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") " pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:51.934183 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:51.934086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:02:52.215397 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:52.215359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"]
Sep 12 02:02:52.216626 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:02:52.216594    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc0ee45f_8f90_428d_a6ca_0911fa9c9a02.slice/crio-6cba9fbc87cce83e3d85916291bba4b1487e0b9e6b3895fb94c5725daf8cc17b WatchSource:0}: Error finding container 6cba9fbc87cce83e3d85916291bba4b1487e0b9e6b3895fb94c5725daf8cc17b: Status 404 returned error can't find the container with id 6cba9fbc87cce83e3d85916291bba4b1487e0b9e6b3895fb94c5725daf8cc17b
Sep 12 02:02:52.362767 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:52.362721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx" event={"ID":"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02","Type":"ContainerStarted","Data":"6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e"}
Sep 12 02:02:52.362934 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:52.362779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx" event={"ID":"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02","Type":"ContainerStarted","Data":"6cba9fbc87cce83e3d85916291bba4b1487e0b9e6b3895fb94c5725daf8cc17b"}
Sep 12 02:02:52.379533 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:02:52.379479    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx" podStartSLOduration=1.379463413 podStartE2EDuration="1.379463413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:02:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:02:52.378579357 +0000 UTC m=+1077.905480437" watchObservedRunningTime="2026-09-12 02:02:52.379463413 +0000 UTC m=+1077.906364492"
Sep 12 02:03:01.935286 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:01.935233    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:03:01.935741 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:01.935304    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:03:01.950027 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:01.949937    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:03:02.442864 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:02.442831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:03:10.923327 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:10.923291    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"]
Sep 12 02:03:10.927077 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:10.927052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:10.929570 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:10.929547    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-98f68e\"/\"e2e-conflict-a-3901f64e-kserve-self-signed-certs\""
Sep 12 02:03:10.937308 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:10.937284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"]
Sep 12 02:03:11.037321 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.037287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qkzt\" (UniqueName: \"kubernetes.io/projected/db65a202-818d-441e-abba-aa801362caf4-kube-api-access-2qkzt\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.037506 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.037405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-dshm\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.037506 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.037447    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-model-cache\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.037634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.037547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-tmp-dir\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.037691 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.037646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-home\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.037744 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.037698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db65a202-818d-441e-abba-aa801362caf4-tls-certs\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.139286 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.139246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db65a202-818d-441e-abba-aa801362caf4-tls-certs\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.139482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.139329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qkzt\" (UniqueName: \"kubernetes.io/projected/db65a202-818d-441e-abba-aa801362caf4-kube-api-access-2qkzt\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.139482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.139349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-dshm\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.139482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.139384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-model-cache\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.139482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.139411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-tmp-dir\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.139482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.139474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-home\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.139878 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.139850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-tmp-dir\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.140018 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.139885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-model-cache\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.140018 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.139919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-home\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.141636 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.141608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-dshm\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.141803 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.141784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db65a202-818d-441e-abba-aa801362caf4-tls-certs\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.148300 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.148279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qkzt\" (UniqueName: \"kubernetes.io/projected/db65a202-818d-441e-abba-aa801362caf4-kube-api-access-2qkzt\") pod \"e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") " pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.237177 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.237092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:11.512329 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:11.512299    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"]
Sep 12 02:03:11.513877 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:03:11.513836    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb65a202_818d_441e_abba_aa801362caf4.slice/crio-b75a4e5e8b17cf7fd357fdd0ae143b11e038db7ee72ace9538e06fd6d0e835eb WatchSource:0}: Error finding container b75a4e5e8b17cf7fd357fdd0ae143b11e038db7ee72ace9538e06fd6d0e835eb: Status 404 returned error can't find the container with id b75a4e5e8b17cf7fd357fdd0ae143b11e038db7ee72ace9538e06fd6d0e835eb
Sep 12 02:03:12.506706 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:12.506665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn" event={"ID":"db65a202-818d-441e-abba-aa801362caf4","Type":"ContainerStarted","Data":"d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644"}
Sep 12 02:03:12.507181 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:12.506729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn" event={"ID":"db65a202-818d-441e-abba-aa801362caf4","Type":"ContainerStarted","Data":"b75a4e5e8b17cf7fd357fdd0ae143b11e038db7ee72ace9538e06fd6d0e835eb"}
Sep 12 02:03:12.526935 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:12.526874    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn" podStartSLOduration=2.526857793 podStartE2EDuration="2.526857793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:03:12.525263051 +0000 UTC m=+1098.052164134" watchObservedRunningTime="2026-09-12 02:03:12.526857793 +0000 UTC m=+1098.053758873"
Sep 12 02:03:14.308805 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.308763    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"]
Sep 12 02:03:14.312956 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.312932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"
Sep 12 02:03:14.315405 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.315383    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:03:14.315405 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.315395    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:03:14.321097 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.321069    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"]
Sep 12 02:03:14.347538 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.347508    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"]
Sep 12 02:03:14.371205 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.371169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac72b4f5-6374-42a4-8410-eb9bf81cac7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz\" (UID: \"ac72b4f5-6374-42a4-8410-eb9bf81cac7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"
Sep 12 02:03:14.472711 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.472671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac72b4f5-6374-42a4-8410-eb9bf81cac7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz\" (UID: \"ac72b4f5-6374-42a4-8410-eb9bf81cac7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"
Sep 12 02:03:14.473336 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.473312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac72b4f5-6374-42a4-8410-eb9bf81cac7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz\" (UID: \"ac72b4f5-6374-42a4-8410-eb9bf81cac7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"
Sep 12 02:03:14.515422 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.515392    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"]
Sep 12 02:03:14.520294 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.519499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"
Sep 12 02:03:14.527299 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.527270    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"]
Sep 12 02:03:14.574273 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.574176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6538264-e238-4bfd-984f-ad6b19b5bf8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r\" (UID: \"e6538264-e238-4bfd-984f-ad6b19b5bf8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"
Sep 12 02:03:14.628175 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.628141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"
Sep 12 02:03:14.675888 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.675851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6538264-e238-4bfd-984f-ad6b19b5bf8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r\" (UID: \"e6538264-e238-4bfd-984f-ad6b19b5bf8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"
Sep 12 02:03:14.676910 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.676875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6538264-e238-4bfd-984f-ad6b19b5bf8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r\" (UID: \"e6538264-e238-4bfd-984f-ad6b19b5bf8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"
Sep 12 02:03:14.833028 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.832936    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"
Sep 12 02:03:14.903440 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:14.903411    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"]
Sep 12 02:03:14.907120 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:03:14.907050    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac72b4f5_6374_42a4_8410_eb9bf81cac7c.slice/crio-6b5d90115272e0b615cf3543c5308cbb5b72712098e81ec718757f61424ec21f WatchSource:0}: Error finding container 6b5d90115272e0b615cf3543c5308cbb5b72712098e81ec718757f61424ec21f: Status 404 returned error can't find the container with id 6b5d90115272e0b615cf3543c5308cbb5b72712098e81ec718757f61424ec21f
Sep 12 02:03:15.111260 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:15.111234    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"]
Sep 12 02:03:15.112824 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:03:15.112798    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6538264_e238_4bfd_984f_ad6b19b5bf8b.slice/crio-8bfe7303e820f7b3cffe3e31cd8cb82d209a842932c30dc277dad409ba5ae190 WatchSource:0}: Error finding container 8bfe7303e820f7b3cffe3e31cd8cb82d209a842932c30dc277dad409ba5ae190: Status 404 returned error can't find the container with id 8bfe7303e820f7b3cffe3e31cd8cb82d209a842932c30dc277dad409ba5ae190
Sep 12 02:03:15.534257 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:15.534202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r" event={"ID":"e6538264-e238-4bfd-984f-ad6b19b5bf8b","Type":"ContainerStarted","Data":"5bb4f89d140d7d1e41aa1c95c5e9d0e23d2cd217c84964dcc4062a1c790bc237"}
Sep 12 02:03:15.534724 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:15.534290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r" event={"ID":"e6538264-e238-4bfd-984f-ad6b19b5bf8b","Type":"ContainerStarted","Data":"8bfe7303e820f7b3cffe3e31cd8cb82d209a842932c30dc277dad409ba5ae190"}
Sep 12 02:03:15.536050 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:15.536023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz" event={"ID":"ac72b4f5-6374-42a4-8410-eb9bf81cac7c","Type":"ContainerStarted","Data":"c9e20595245b4c5cd2a9098d320319708f5dfbe7192e50da919e0e5b672a6556"}
Sep 12 02:03:15.536171 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:15.536076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz" event={"ID":"ac72b4f5-6374-42a4-8410-eb9bf81cac7c","Type":"ContainerStarted","Data":"6b5d90115272e0b615cf3543c5308cbb5b72712098e81ec718757f61424ec21f"}
Sep 12 02:03:15.549907 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:15.549849    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r" podStartSLOduration=1.549832942 podStartE2EDuration="1.549832942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:03:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:03:15.547797081 +0000 UTC m=+1101.074698162" watchObservedRunningTime="2026-09-12 02:03:15.549832942 +0000 UTC m=+1101.076734022"
Sep 12 02:03:15.565417 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:15.565364    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz" podStartSLOduration=1.565349503 podStartE2EDuration="1.565349503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:03:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:03:15.564461996 +0000 UTC m=+1101.091363076" watchObservedRunningTime="2026-09-12 02:03:15.565349503 +0000 UTC m=+1101.092250582"
Sep 12 02:03:16.544283 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:16.544243    2580 generic.go:356] "Generic (PLEG): container finished" podID="e6538264-e238-4bfd-984f-ad6b19b5bf8b" containerID="5bb4f89d140d7d1e41aa1c95c5e9d0e23d2cd217c84964dcc4062a1c790bc237" exitCode=0
Sep 12 02:03:16.544713 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:16.544339    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r" event={"ID":"e6538264-e238-4bfd-984f-ad6b19b5bf8b","Type":"ContainerDied","Data":"5bb4f89d140d7d1e41aa1c95c5e9d0e23d2cd217c84964dcc4062a1c790bc237"}
Sep 12 02:03:16.546035 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:16.546001    2580 generic.go:356] "Generic (PLEG): container finished" podID="ac72b4f5-6374-42a4-8410-eb9bf81cac7c" containerID="c9e20595245b4c5cd2a9098d320319708f5dfbe7192e50da919e0e5b672a6556" exitCode=0
Sep 12 02:03:16.546164 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:16.546052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz" event={"ID":"ac72b4f5-6374-42a4-8410-eb9bf81cac7c","Type":"ContainerDied","Data":"c9e20595245b4c5cd2a9098d320319708f5dfbe7192e50da919e0e5b672a6556"}
Sep 12 02:03:17.696497 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:17.696460    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"
Sep 12 02:03:17.731769 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:17.731741    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"
Sep 12 02:03:17.808019 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:17.807915    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6538264-e238-4bfd-984f-ad6b19b5bf8b-maas-api-service-ca\") pod \"e6538264-e238-4bfd-984f-ad6b19b5bf8b\" (UID: \"e6538264-e238-4bfd-984f-ad6b19b5bf8b\") "
Sep 12 02:03:17.808200 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:17.808079    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac72b4f5-6374-42a4-8410-eb9bf81cac7c-maas-api-service-ca\") pod \"ac72b4f5-6374-42a4-8410-eb9bf81cac7c\" (UID: \"ac72b4f5-6374-42a4-8410-eb9bf81cac7c\") "
Sep 12 02:03:17.808315 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:17.808289    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6538264-e238-4bfd-984f-ad6b19b5bf8b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6538264-e238-4bfd-984f-ad6b19b5bf8b" (UID: "e6538264-e238-4bfd-984f-ad6b19b5bf8b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:03:17.808450 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:17.808428    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac72b4f5-6374-42a4-8410-eb9bf81cac7c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ac72b4f5-6374-42a4-8410-eb9bf81cac7c" (UID: "ac72b4f5-6374-42a4-8410-eb9bf81cac7c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:03:17.808519 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:17.808508    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac72b4f5-6374-42a4-8410-eb9bf81cac7c-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:17.808558 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:17.808522    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6538264-e238-4bfd-984f-ad6b19b5bf8b-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:18.560151 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:18.560116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r" event={"ID":"e6538264-e238-4bfd-984f-ad6b19b5bf8b","Type":"ContainerDied","Data":"8bfe7303e820f7b3cffe3e31cd8cb82d209a842932c30dc277dad409ba5ae190"}
Sep 12 02:03:18.560379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:18.560177    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bfe7303e820f7b3cffe3e31cd8cb82d209a842932c30dc277dad409ba5ae190"
Sep 12 02:03:18.560379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:18.560120    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"
Sep 12 02:03:18.561660 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:18.561639    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"
Sep 12 02:03:18.561778 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:18.561659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz" event={"ID":"ac72b4f5-6374-42a4-8410-eb9bf81cac7c","Type":"ContainerDied","Data":"6b5d90115272e0b615cf3543c5308cbb5b72712098e81ec718757f61424ec21f"}
Sep 12 02:03:18.561778 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:18.561706    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b5d90115272e0b615cf3543c5308cbb5b72712098e81ec718757f61424ec21f"
Sep 12 02:03:20.019600 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.019565    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"]
Sep 12 02:03:20.020230 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.020199    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac72b4f5-6374-42a4-8410-eb9bf81cac7c" containerName="revoke-keys"
Sep 12 02:03:20.020286 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.020235    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac72b4f5-6374-42a4-8410-eb9bf81cac7c" containerName="revoke-keys"
Sep 12 02:03:20.020286 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.020270    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6538264-e238-4bfd-984f-ad6b19b5bf8b" containerName="revoke-keys"
Sep 12 02:03:20.020286 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.020279    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6538264-e238-4bfd-984f-ad6b19b5bf8b" containerName="revoke-keys"
Sep 12 02:03:20.020422 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.020410    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6538264-e238-4bfd-984f-ad6b19b5bf8b" containerName="revoke-keys"
Sep 12 02:03:20.020457 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.020427    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac72b4f5-6374-42a4-8410-eb9bf81cac7c" containerName="revoke-keys"
Sep 12 02:03:20.024093 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.024073    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.026892 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.026860    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f660a6\"/\"distinct-w0-f660a6-kserve-self-signed-certs\""
Sep 12 02:03:20.033418 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.033394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"]
Sep 12 02:03:20.130372 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.130331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg2m9\" (UniqueName: \"kubernetes.io/projected/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-kube-api-access-tg2m9\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.130372 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.130377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-dshm\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.130611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.130401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-model-cache\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.130611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.130451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-home\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.130611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.130540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tmp-dir\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.130611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.130557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tls-certs\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.231763 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.231720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg2m9\" (UniqueName: \"kubernetes.io/projected/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-kube-api-access-tg2m9\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.231763 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.231764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-dshm\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.232051 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.231885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-model-cache\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.232051 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.231912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-home\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.232157 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.232087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tmp-dir\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.232157 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.232114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tls-certs\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.232380 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.232358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-home\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.232457 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.232384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-model-cache\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.232516 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.232483    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tmp-dir\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.234488 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.234467    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-dshm\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.234718 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.234699    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tls-certs\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.239656 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.239636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg2m9\" (UniqueName: \"kubernetes.io/projected/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-kube-api-access-tg2m9\") pod \"distinct-w0-f660a6-kserve-58646b5978-6d4rq\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") " pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.334864 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.334767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:20.613303 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:20.613271    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"]
Sep 12 02:03:20.614050 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:03:20.614024    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93407ba5_d5a5_49c5_8ad4_371a44c1d45c.slice/crio-aee3570ef01ee3bd4baeef98d06540718715cac52f1fa8e3e8973247f8085c69 WatchSource:0}: Error finding container aee3570ef01ee3bd4baeef98d06540718715cac52f1fa8e3e8973247f8085c69: Status 404 returned error can't find the container with id aee3570ef01ee3bd4baeef98d06540718715cac52f1fa8e3e8973247f8085c69
Sep 12 02:03:21.237746 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:21.237705    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:21.238128 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:21.237768    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:21.251249 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:21.251202    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:21.586304 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:21.586264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq" event={"ID":"93407ba5-d5a5-49c5-8ad4-371a44c1d45c","Type":"ContainerStarted","Data":"51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84"}
Sep 12 02:03:21.586304 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:21.586312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq" event={"ID":"93407ba5-d5a5-49c5-8ad4-371a44c1d45c","Type":"ContainerStarted","Data":"aee3570ef01ee3bd4baeef98d06540718715cac52f1fa8e3e8973247f8085c69"}
Sep 12 02:03:21.598272 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:21.598243    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:03:21.605030 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:21.604987    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq" podStartSLOduration=2.604972772 podStartE2EDuration="2.604972772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:03:21.604121347 +0000 UTC m=+1107.131022427" watchObservedRunningTime="2026-09-12 02:03:21.604972772 +0000 UTC m=+1107.131873852"
Sep 12 02:03:24.754116 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:24.754076    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"]
Sep 12 02:03:24.755768 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:24.755477    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d" podUID="fd1243ae-58ad-419e-a1ff-320d9e9ccf37" containerName="payload-processing" containerID="cri-o://7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd" gracePeriod=30
Sep 12 02:03:24.936340 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:24.936280    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"]
Sep 12 02:03:24.939258 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:24.939229    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-b838b7-tbh4r"]
Sep 12 02:03:24.983158 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:24.983128    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6538264-e238-4bfd-984f-ad6b19b5bf8b" path="/var/lib/kubelet/pods/e6538264-e238-4bfd-984f-ad6b19b5bf8b/volumes"
Sep 12 02:03:25.033677 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.033649    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:03:25.181037 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.181005    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmxb8\" (UniqueName: \"kubernetes.io/projected/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-kube-api-access-wmxb8\") pod \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\" (UID: \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\") "
Sep 12 02:03:25.181037 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.181042    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-plugins-config-volume\") pod \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\" (UID: \"fd1243ae-58ad-419e-a1ff-320d9e9ccf37\") "
Sep 12 02:03:25.181508 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.181480    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd1243ae-58ad-419e-a1ff-320d9e9ccf37" (UID: "fd1243ae-58ad-419e-a1ff-320d9e9ccf37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:03:25.183203 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.183179    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-kube-api-access-wmxb8" (OuterVolumeSpecName: "kube-api-access-wmxb8") pod "fd1243ae-58ad-419e-a1ff-320d9e9ccf37" (UID: "fd1243ae-58ad-419e-a1ff-320d9e9ccf37"). InnerVolumeSpecName "kube-api-access-wmxb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:03:25.282010 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.281914    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmxb8\" (UniqueName: \"kubernetes.io/projected/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-kube-api-access-wmxb8\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:25.282010 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.281947    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1243ae-58ad-419e-a1ff-320d9e9ccf37-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:25.288831 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.288799    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"]
Sep 12 02:03:25.291722 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.291696    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d5764ff8-7g5pz"]
Sep 12 02:03:25.615793 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.615704    2580 generic.go:356] "Generic (PLEG): container finished" podID="fd1243ae-58ad-419e-a1ff-320d9e9ccf37" containerID="7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd" exitCode=0
Sep 12 02:03:25.615793 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.615745    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"
Sep 12 02:03:25.615977 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.615786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d" event={"ID":"fd1243ae-58ad-419e-a1ff-320d9e9ccf37","Type":"ContainerDied","Data":"7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd"}
Sep 12 02:03:25.615977 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.615845    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d" event={"ID":"fd1243ae-58ad-419e-a1ff-320d9e9ccf37","Type":"ContainerDied","Data":"a536aa9aa96439b73493cfd5c89ea4a3d8b38d7dbae57671f3e41c5db644d650"}
Sep 12 02:03:25.615977 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.615876    2580 scope.go:117] "RemoveContainer" containerID="7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd"
Sep 12 02:03:25.625806 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.625788    2580 scope.go:117] "RemoveContainer" containerID="7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd"
Sep 12 02:03:25.626123 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:03:25.626105    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd\": container with ID starting with 7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd not found: ID does not exist" containerID="7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd"
Sep 12 02:03:25.626197 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.626136    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd"} err="failed to get container status \"7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd\": rpc error: code = NotFound desc = could not find container \"7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd\": container with ID starting with 7bc91d98e7d634cdc64847d38d4c9ef1249cbbd171cc01d05c9d3c2df2e9e1dd not found: ID does not exist"
Sep 12 02:03:25.638974 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.638944    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"]
Sep 12 02:03:25.641884 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:25.641858    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b838b7-5846887b89-nqg8d"]
Sep 12 02:03:26.980550 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:26.980516    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac72b4f5-6374-42a4-8410-eb9bf81cac7c" path="/var/lib/kubelet/pods/ac72b4f5-6374-42a4-8410-eb9bf81cac7c/volumes"
Sep 12 02:03:26.980930 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:26.980909    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd1243ae-58ad-419e-a1ff-320d9e9ccf37" path="/var/lib/kubelet/pods/fd1243ae-58ad-419e-a1ff-320d9e9ccf37/volumes"
Sep 12 02:03:29.369673 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.369618    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx" podUID="bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" containerName="main" containerID="cri-o://6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e" gracePeriod=45
Sep 12 02:03:29.621006 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.620933    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:03:29.646847 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.646812    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" containerID="6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e" exitCode=0
Sep 12 02:03:29.646998 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.646875    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"
Sep 12 02:03:29.646998 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.646901    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx" event={"ID":"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02","Type":"ContainerDied","Data":"6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e"}
Sep 12 02:03:29.646998 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.646955    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx" event={"ID":"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02","Type":"ContainerDied","Data":"6cba9fbc87cce83e3d85916291bba4b1487e0b9e6b3895fb94c5725daf8cc17b"}
Sep 12 02:03:29.646998 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.646983    2580 scope.go:117] "RemoveContainer" containerID="6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e"
Sep 12 02:03:29.659055 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.659031    2580 scope.go:117] "RemoveContainer" containerID="6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e"
Sep 12 02:03:29.659417 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:03:29.659392    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e\": container with ID starting with 6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e not found: ID does not exist" containerID="6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e"
Sep 12 02:03:29.659532 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.659424    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e"} err="failed to get container status \"6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e\": rpc error: code = NotFound desc = could not find container \"6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e\": container with ID starting with 6e11166ab747aea41d9804d95264a381adc903007a440cb63d2db75ad829a56e not found: ID does not exist"
Sep 12 02:03:29.737982 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.737943    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtf57\" (UniqueName: \"kubernetes.io/projected/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-kube-api-access-jtf57\") pod \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") "
Sep 12 02:03:29.738181 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.737997    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-model-cache\") pod \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") "
Sep 12 02:03:29.738181 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738023    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-dshm\") pod \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") "
Sep 12 02:03:29.738181 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738057    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tmp-dir\") pod \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") "
Sep 12 02:03:29.738181 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738120    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-home\") pod \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") "
Sep 12 02:03:29.738436 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738201    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tls-certs\") pod \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\" (UID: \"bc0ee45f-8f90-428d-a6ca-0911fa9c9a02\") "
Sep 12 02:03:29.738436 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738278    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-model-cache" (OuterVolumeSpecName: "model-cache") pod "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" (UID: "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:03:29.738436 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738394    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" (UID: "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:03:29.738568 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738474    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-home" (OuterVolumeSpecName: "home") pod "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" (UID: "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:03:29.738712 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738694    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:29.738785 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738714    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:29.738785 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.738728    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:29.740377 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.740355    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-dshm" (OuterVolumeSpecName: "dshm") pod "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" (UID: "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:03:29.740489 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.740411    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-kube-api-access-jtf57" (OuterVolumeSpecName: "kube-api-access-jtf57") pod "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" (UID: "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02"). InnerVolumeSpecName "kube-api-access-jtf57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:03:29.740489 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.740464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" (UID: "bc0ee45f-8f90-428d-a6ca-0911fa9c9a02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:03:29.840326 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.840282    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtf57\" (UniqueName: \"kubernetes.io/projected/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-kube-api-access-jtf57\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:29.840326 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.840320    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:29.840326 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.840331    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:29.971506 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.971474    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"]
Sep 12 02:03:29.976390 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:29.976360    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-b838b7/explicit-model-b838b7-kserve-65db7c769c-nljpx"]
Sep 12 02:03:30.334937 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:30.334892    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:30.335155 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:30.334952    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:30.350600 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:30.350560    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:30.667013 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:30.666917    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:03:30.979946 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:30.979844    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" path="/var/lib/kubelet/pods/bc0ee45f-8f90-428d-a6ca-0911fa9c9a02/volumes"
Sep 12 02:03:31.577513 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:31.577465    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"]
Sep 12 02:03:31.577764 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:31.577737    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy" containerID="cri-o://b755e45cca2c8cf702fe3764747f88045d64f05771895e4d070ffe9f2cfa46f0" gracePeriod=30
Sep 12 02:03:36.703028 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:36.702992    2580 generic.go:356] "Generic (PLEG): container finished" podID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerID="b755e45cca2c8cf702fe3764747f88045d64f05771895e4d070ffe9f2cfa46f0" exitCode=0
Sep 12 02:03:36.703390 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:36.703045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" event={"ID":"882c1f21-e304-4ca3-8e6c-bbac21cede11","Type":"ContainerDied","Data":"b755e45cca2c8cf702fe3764747f88045d64f05771895e4d070ffe9f2cfa46f0"}
Sep 12 02:03:36.703390 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:36.703110    2580 scope.go:117] "RemoveContainer" containerID="30688fc294c6c445be2d43fc2bd83d2a9bcd47521716d0b37ac1cd413a8f5e2f"
Sep 12 02:03:36.851337 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:36.851268    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:03:37.016088 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016048    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn6pw\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-kube-api-access-fn6pw\") pod \"882c1f21-e304-4ca3-8e6c-bbac21cede11\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") "
Sep 12 02:03:37.016335 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882c1f21-e304-4ca3-8e6c-bbac21cede11-istiod-ca-cert\") pod \"882c1f21-e304-4ca3-8e6c-bbac21cede11\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") "
Sep 12 02:03:37.016335 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016183    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-envoy\") pod \"882c1f21-e304-4ca3-8e6c-bbac21cede11\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") "
Sep 12 02:03:37.016335 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016276    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-podinfo\") pod \"882c1f21-e304-4ca3-8e6c-bbac21cede11\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") "
Sep 12 02:03:37.016335 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016302    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-credential-socket\") pod \"882c1f21-e304-4ca3-8e6c-bbac21cede11\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") "
Sep 12 02:03:37.016335 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016327    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-certs\") pod \"882c1f21-e304-4ca3-8e6c-bbac21cede11\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") "
Sep 12 02:03:37.016619 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016356    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-data\") pod \"882c1f21-e304-4ca3-8e6c-bbac21cede11\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") "
Sep 12 02:03:37.016619 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016407    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-token\") pod \"882c1f21-e304-4ca3-8e6c-bbac21cede11\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") "
Sep 12 02:03:37.016619 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016436    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-socket\") pod \"882c1f21-e304-4ca3-8e6c-bbac21cede11\" (UID: \"882c1f21-e304-4ca3-8e6c-bbac21cede11\") "
Sep 12 02:03:37.016619 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016586    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/882c1f21-e304-4ca3-8e6c-bbac21cede11-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "882c1f21-e304-4ca3-8e6c-bbac21cede11" (UID: "882c1f21-e304-4ca3-8e6c-bbac21cede11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:03:37.016814 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016660    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "882c1f21-e304-4ca3-8e6c-bbac21cede11" (UID: "882c1f21-e304-4ca3-8e6c-bbac21cede11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:03:37.016814 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016697    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "882c1f21-e304-4ca3-8e6c-bbac21cede11" (UID: "882c1f21-e304-4ca3-8e6c-bbac21cede11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:03:37.016923 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016826    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "882c1f21-e304-4ca3-8e6c-bbac21cede11" (UID: "882c1f21-e304-4ca3-8e6c-bbac21cede11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:03:37.016923 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-data" (OuterVolumeSpecName: "istio-data") pod "882c1f21-e304-4ca3-8e6c-bbac21cede11" (UID: "882c1f21-e304-4ca3-8e6c-bbac21cede11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:03:37.017030 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016938    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:37.017030 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016955    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:37.017030 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016970    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:37.017030 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.016983    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882c1f21-e304-4ca3-8e6c-bbac21cede11-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:37.018540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.018515    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "882c1f21-e304-4ca3-8e6c-bbac21cede11" (UID: "882c1f21-e304-4ca3-8e6c-bbac21cede11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:03:37.018749 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.018725    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-kube-api-access-fn6pw" (OuterVolumeSpecName: "kube-api-access-fn6pw") pod "882c1f21-e304-4ca3-8e6c-bbac21cede11" (UID: "882c1f21-e304-4ca3-8e6c-bbac21cede11"). InnerVolumeSpecName "kube-api-access-fn6pw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:03:37.018749 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.018733    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "882c1f21-e304-4ca3-8e6c-bbac21cede11" (UID: "882c1f21-e304-4ca3-8e6c-bbac21cede11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:03:37.018854 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.018732    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-token" (OuterVolumeSpecName: "istio-token") pod "882c1f21-e304-4ca3-8e6c-bbac21cede11" (UID: "882c1f21-e304-4ca3-8e6c-bbac21cede11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:03:37.118382 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.118297    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:37.118382 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.118322    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:37.118382 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.118331    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:37.118382 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.118340    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:37.118382 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.118349    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn6pw\" (UniqueName: \"kubernetes.io/projected/882c1f21-e304-4ca3-8e6c-bbac21cede11-kube-api-access-fn6pw\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:03:37.710691 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.710659    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"
Sep 12 02:03:37.711192 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.710706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs" event={"ID":"882c1f21-e304-4ca3-8e6c-bbac21cede11","Type":"ContainerDied","Data":"da0d6d63de1190c85323692d75b7b4eb3933f8a30cffea90f0d2037949ff6fa8"}
Sep 12 02:03:37.711192 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.710772    2580 scope.go:117] "RemoveContainer" containerID="b755e45cca2c8cf702fe3764747f88045d64f05771895e4d070ffe9f2cfa46f0"
Sep 12 02:03:37.735610 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.735581    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"]
Sep 12 02:03:37.738097 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:37.738074    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-b838b7-openshift-default-549c4c6-ctzgs"]
Sep 12 02:03:38.979930 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:38.979896    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" path="/var/lib/kubelet/pods/882c1f21-e304-4ca3-8e6c-bbac21cede11/volumes"
Sep 12 02:03:46.065140 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065098    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"]
Sep 12 02:03:46.065747 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065729    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy"
Sep 12 02:03:46.065791 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065751    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy"
Sep 12 02:03:46.065791 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065767    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" containerName="main"
Sep 12 02:03:46.065791 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065777    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" containerName="main"
Sep 12 02:03:46.065887 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065796    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd1243ae-58ad-419e-a1ff-320d9e9ccf37" containerName="payload-processing"
Sep 12 02:03:46.065887 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065806    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd1243ae-58ad-419e-a1ff-320d9e9ccf37" containerName="payload-processing"
Sep 12 02:03:46.065887 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065828    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy"
Sep 12 02:03:46.065887 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065837    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy"
Sep 12 02:03:46.066005 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065928    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy"
Sep 12 02:03:46.066005 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065938    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="882c1f21-e304-4ca3-8e6c-bbac21cede11" containerName="istio-proxy"
Sep 12 02:03:46.066005 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065949    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd1243ae-58ad-419e-a1ff-320d9e9ccf37" containerName="payload-processing"
Sep 12 02:03:46.066005 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.065960    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc0ee45f-8f90-428d-a6ca-0911fa9c9a02" containerName="main"
Sep 12 02:03:46.071141 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.071115    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.074130 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.074106    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-e8046f-openshift-default-dockercfg-74lcv\""
Sep 12 02:03:46.083548 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.083521    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"]
Sep 12 02:03:46.115045 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.115017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-podinfo\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.115161 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.115053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-token\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.115161 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.115133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istiod-ca-cert\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.115280 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.115205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-envoy\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.115280 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.115248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-credential-socket\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.115347 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.115280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-socket\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.115347 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.115312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-certs\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.115414 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.115352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq2gh\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-kube-api-access-mq2gh\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.115414 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.115395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-data\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216103 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-socket\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216103 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-certs\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216338 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq2gh\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-kube-api-access-mq2gh\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216338 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-data\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216446 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-podinfo\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216446 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-token\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216560 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istiod-ca-cert\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216560 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216485    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-certs\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216560 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-socket\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216560 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-envoy\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216787 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-credential-socket\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216787 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-data\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.216930 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.216912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-credential-socket\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.217086 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.217067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istiod-ca-cert\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.218713 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.218691    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-envoy\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.218932 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.218915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-podinfo\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.224731 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.224708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq2gh\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-kube-api-access-mq2gh\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.224988 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.224969    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-token\") pod \"e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") " pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.383652 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.383549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:46.661643 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.661618    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"]
Sep 12 02:03:46.664438 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:03:46.664401    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ed9705_6d07_457a_aa38_8c4e3d9a68dc.slice/crio-b7319fdfd27dec8435300d12625824f61b87b6edb857069324744bd262a30dea WatchSource:0}: Error finding container b7319fdfd27dec8435300d12625824f61b87b6edb857069324744bd262a30dea: Status 404 returned error can't find the container with id b7319fdfd27dec8435300d12625824f61b87b6edb857069324744bd262a30dea
Sep 12 02:03:46.667192 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.667152    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:03:46.667327 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.667257    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:03:46.667327 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.667312    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:03:46.778269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.778226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" event={"ID":"58ed9705-6d07-457a-aa38-8c4e3d9a68dc","Type":"ContainerStarted","Data":"e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284"}
Sep 12 02:03:46.778445 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.778297    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" event={"ID":"58ed9705-6d07-457a-aa38-8c4e3d9a68dc","Type":"ContainerStarted","Data":"b7319fdfd27dec8435300d12625824f61b87b6edb857069324744bd262a30dea"}
Sep 12 02:03:46.800431 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:46.800375    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" podStartSLOduration=0.800359093 podStartE2EDuration="800.359093ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:03:46.79776006 +0000 UTC m=+1132.324661140" watchObservedRunningTime="2026-09-12 02:03:46.800359093 +0000 UTC m=+1132.327260172"
Sep 12 02:03:47.384383 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:47.384350    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:47.389646 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:47.389618    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:47.784595 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:47.784559    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:03:47.785649 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:03:47.785629    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:04:10.951339 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:10.951238    2580 generic.go:356] "Generic (PLEG): container finished" podID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerID="e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284" exitCode=0
Sep 12 02:04:10.951339 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:10.951299    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" event={"ID":"58ed9705-6d07-457a-aa38-8c4e3d9a68dc","Type":"ContainerDied","Data":"e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284"}
Sep 12 02:04:10.951846 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:10.951735    2580 scope.go:117] "RemoveContainer" containerID="e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284"
Sep 12 02:04:10.952862 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:10.952837    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:04:10.952921 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:10.952908    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:04:10.952957 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:10.952943    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:04:11.012555 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.012525    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"]
Sep 12 02:04:11.015776 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.015757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.018458 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.018332    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-b99e21\"/\"kube-root-ca.crt\""
Sep 12 02:04:11.018458 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.018370    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-b99e21\"/\"noten-model-b99e21-kserve-self-signed-certs\""
Sep 12 02:04:11.018458 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.018340    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-b99e21\"/\"default-dockercfg-8lnw7\""
Sep 12 02:04:11.018458 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.018439    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-b99e21\"/\"openshift-service-ca.crt\""
Sep 12 02:04:11.029040 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.029007    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"]
Sep 12 02:04:11.182404 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.182368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-dshm\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.182606 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.182437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhn4s\" (UniqueName: \"kubernetes.io/projected/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-kube-api-access-zhn4s\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.182606 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.182547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-home\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.182748 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.182602    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tmp-dir\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.182748 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.182678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tls-certs\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.182748 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.182738    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-model-cache\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.284457 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.284405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-dshm\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.284660 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.284489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhn4s\" (UniqueName: \"kubernetes.io/projected/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-kube-api-access-zhn4s\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.284660 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.284573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-home\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.284660 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.284613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tmp-dir\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.284853 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.284691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tls-certs\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.284853 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.284728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-model-cache\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.285083 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.285055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-home\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.285164 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.285127    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tmp-dir\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.285320 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.285298    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-model-cache\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.287436 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.287391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-dshm\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.288114 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.288091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tls-certs\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.294647 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.294590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhn4s\" (UniqueName: \"kubernetes.io/projected/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-kube-api-access-zhn4s\") pod \"noten-model-b99e21-kserve-69759ff699-pqgnw\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") " pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.331489 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.331455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:11.617127 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.617099    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"]
Sep 12 02:04:11.617968 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:04:11.617944    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff7bc83b_6cb9_49db_add4_16bbb2a719ef.slice/crio-2a7162f5eba932a2b067b4f70855ffcf4db3742c7d927f10f9a528442eaff418 WatchSource:0}: Error finding container 2a7162f5eba932a2b067b4f70855ffcf4db3742c7d927f10f9a528442eaff418: Status 404 returned error can't find the container with id 2a7162f5eba932a2b067b4f70855ffcf4db3742c7d927f10f9a528442eaff418
Sep 12 02:04:11.960233 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.960118    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" event={"ID":"58ed9705-6d07-457a-aa38-8c4e3d9a68dc","Type":"ContainerStarted","Data":"d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9"}
Sep 12 02:04:11.961942 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.961905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw" event={"ID":"ff7bc83b-6cb9-49db-add4-16bbb2a719ef","Type":"ContainerStarted","Data":"9cbf08bb0229e5f65b137c4a45251d4d78889daf25153b7d98168a6276f31b03"}
Sep 12 02:04:11.962076 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:11.961949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw" event={"ID":"ff7bc83b-6cb9-49db-add4-16bbb2a719ef","Type":"ContainerStarted","Data":"2a7162f5eba932a2b067b4f70855ffcf4db3742c7d927f10f9a528442eaff418"}
Sep 12 02:04:12.003951 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:12.003899    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw" podStartSLOduration=2.003884334 podStartE2EDuration="2.003884334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:04:12.003319613 +0000 UTC m=+1157.530220694" watchObservedRunningTime="2026-09-12 02:04:12.003884334 +0000 UTC m=+1157.530785414"
Sep 12 02:04:12.384324 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:12.384280    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 12 02:04:12.384535 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:12.384348    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 12 02:04:12.384651 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:12.384631    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:04:13.384846 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:13.384806    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 12 02:04:13.385456 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:13.385419    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 12 02:04:14.384924 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.384882    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 12 02:04:14.385550 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.385516    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 12 02:04:14.507170 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.505536    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"]
Sep 12 02:04:14.509301 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.509276    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.510620 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.510563    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"]
Sep 12 02:04:14.512589 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.512564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-ac814d-openshift-default-dockercfg-8l8h2\""
Sep 12 02:04:14.518275 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.518030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-socket\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.518275 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.518171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-podinfo\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.518275 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.518202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-certs\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.518275 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.518255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-credential-socket\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.518522 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.518319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rc59\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-kube-api-access-6rc59\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.518522 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.518373    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-data\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.518522 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.518411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-token\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.518522 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.518440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-envoy\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.518522 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.518504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6bdc5c-073f-4328-8d5f-612be56daa64-istiod-ca-cert\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.619846 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.619809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-podinfo\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.619846 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.619851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-certs\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620080 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.619969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-credential-socket\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620080 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rc59\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-kube-api-access-6rc59\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620080 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-data\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620257 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-token\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620257 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-envoy\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620257 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6bdc5c-073f-4328-8d5f-612be56daa64-istiod-ca-cert\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620257 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620242    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-certs\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620518 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620304    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-socket\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620518 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-credential-socket\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620518 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-data\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.620661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620641    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-socket\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.621015 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.620999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6bdc5c-073f-4328-8d5f-612be56daa64-istiod-ca-cert\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.622314 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.622283    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-envoy\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.622434 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.622326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-podinfo\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.628076 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.628049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rc59\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-kube-api-access-6rc59\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.628385 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.628368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-token\") pod \"e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") " pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:14.822682 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:14.822639    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:15.132995 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:15.132969    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"]
Sep 12 02:04:15.136914 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:04:15.136449    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a6bdc5c_073f_4328_8d5f_612be56daa64.slice/crio-c8fd66306ffd542c96b5ecaec40b3eaca2eebf937a6649ec83df813d21bd8b29 WatchSource:0}: Error finding container c8fd66306ffd542c96b5ecaec40b3eaca2eebf937a6649ec83df813d21bd8b29: Status 404 returned error can't find the container with id c8fd66306ffd542c96b5ecaec40b3eaca2eebf937a6649ec83df813d21bd8b29
Sep 12 02:04:15.139080 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:15.139036    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:04:15.139204 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:15.139159    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:04:15.139292 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:15.139202    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:04:15.384923 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:15.384824    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 12 02:04:15.385190 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:15.385159    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 12 02:04:16.012124 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:16.012088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" event={"ID":"3a6bdc5c-073f-4328-8d5f-612be56daa64","Type":"ContainerStarted","Data":"700714d18cd4069fb47040744cd2f9a7b3c6f0d5b2b9c965e2b4f8262ab0ebe1"}
Sep 12 02:04:16.012332 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:16.012142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" event={"ID":"3a6bdc5c-073f-4328-8d5f-612be56daa64","Type":"ContainerStarted","Data":"c8fd66306ffd542c96b5ecaec40b3eaca2eebf937a6649ec83df813d21bd8b29"}
Sep 12 02:04:16.038377 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:16.038319    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" podStartSLOduration=2.038297089 podStartE2EDuration="2.038297089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:04:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:04:16.037476372 +0000 UTC m=+1161.564377453" watchObservedRunningTime="2026-09-12 02:04:16.038297089 +0000 UTC m=+1161.565198170"
Sep 12 02:04:16.384322 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:16.384202    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 12 02:04:16.384322 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:16.384300    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 12 02:04:16.822923 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:16.822883    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:16.828784 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:16.828756    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:17.021092 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:17.021057    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:17.022423 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:17.022399    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:17.388793 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:17.388709    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:04:17.389100 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:17.389081    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:04:17.389885 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:17.389862    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:04:21.331973 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:21.331941    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:21.332392 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:21.331988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:21.344357 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:21.344328    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:22.065996 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:22.065963    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:04:40.094928 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.094890    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"]
Sep 12 02:04:40.098438 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.098414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"
Sep 12 02:04:40.100748 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.100729    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:04:40.100847 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.100791    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:04:40.106313 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.106287    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"]
Sep 12 02:04:40.196538 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.196505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3477a257-c705-4c61-85bd-aac544455bb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq\" (UID: \"3477a257-c705-4c61-85bd-aac544455bb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"
Sep 12 02:04:40.297354 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.297319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3477a257-c705-4c61-85bd-aac544455bb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq\" (UID: \"3477a257-c705-4c61-85bd-aac544455bb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"
Sep 12 02:04:40.297957 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.297937    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3477a257-c705-4c61-85bd-aac544455bb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq\" (UID: \"3477a257-c705-4c61-85bd-aac544455bb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"
Sep 12 02:04:40.412788 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.412694    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"
Sep 12 02:04:40.697549 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:40.697455    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"]
Sep 12 02:04:41.184361 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:41.184311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq" event={"ID":"3477a257-c705-4c61-85bd-aac544455bb4","Type":"ContainerStarted","Data":"6f435e850d4866de389028fac4cbe789b4a718309c4c4942928822ea75a91370"}
Sep 12 02:04:41.184750 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:41.184378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq" event={"ID":"3477a257-c705-4c61-85bd-aac544455bb4","Type":"ContainerStarted","Data":"972e95df4ea4cf13b213652c6e84da9c04f57831268c342904416b7af34ac2db"}
Sep 12 02:04:41.201628 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:41.201574    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq" podStartSLOduration=1.201556333 podStartE2EDuration="1.201556333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:04:41.199802551 +0000 UTC m=+1186.726703641" watchObservedRunningTime="2026-09-12 02:04:41.201556333 +0000 UTC m=+1186.728457500"
Sep 12 02:04:42.192129 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:42.192095    2580 generic.go:356] "Generic (PLEG): container finished" podID="3477a257-c705-4c61-85bd-aac544455bb4" containerID="6f435e850d4866de389028fac4cbe789b4a718309c4c4942928822ea75a91370" exitCode=0
Sep 12 02:04:42.192613 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:42.192183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq" event={"ID":"3477a257-c705-4c61-85bd-aac544455bb4","Type":"ContainerDied","Data":"6f435e850d4866de389028fac4cbe789b4a718309c4c4942928822ea75a91370"}
Sep 12 02:04:43.342488 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:43.342458    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"
Sep 12 02:04:43.533944 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:43.533911    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3477a257-c705-4c61-85bd-aac544455bb4-maas-api-service-ca\") pod \"3477a257-c705-4c61-85bd-aac544455bb4\" (UID: \"3477a257-c705-4c61-85bd-aac544455bb4\") "
Sep 12 02:04:43.534296 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:43.534275    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3477a257-c705-4c61-85bd-aac544455bb4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3477a257-c705-4c61-85bd-aac544455bb4" (UID: "3477a257-c705-4c61-85bd-aac544455bb4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:04:43.635275 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:43.635244    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3477a257-c705-4c61-85bd-aac544455bb4-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:04:44.206980 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:44.206945    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"
Sep 12 02:04:44.207133 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:44.206984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq" event={"ID":"3477a257-c705-4c61-85bd-aac544455bb4","Type":"ContainerDied","Data":"972e95df4ea4cf13b213652c6e84da9c04f57831268c342904416b7af34ac2db"}
Sep 12 02:04:44.207133 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:44.207040    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="972e95df4ea4cf13b213652c6e84da9c04f57831268c342904416b7af34ac2db"
Sep 12 02:04:50.647181 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:50.647037    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"]
Sep 12 02:04:50.647648 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:50.647360    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s" podUID="2e36106c-a86c-47e3-9ad5-3d179946557b" containerName="payload-pre-processing" containerID="cri-o://399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38" gracePeriod=30
Sep 12 02:04:50.799806 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:50.799736    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"]
Sep 12 02:04:50.803993 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:50.803954    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3fc028-t7szq"]
Sep 12 02:04:50.938308 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:50.938281    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 02:04:50.983193 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:50.983153    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3477a257-c705-4c61-85bd-aac544455bb4" path="/var/lib/kubelet/pods/3477a257-c705-4c61-85bd-aac544455bb4/volumes"
Sep 12 02:04:51.020758 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.020727    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e36106c-a86c-47e3-9ad5-3d179946557b-plugins-config-volume\") pod \"2e36106c-a86c-47e3-9ad5-3d179946557b\" (UID: \"2e36106c-a86c-47e3-9ad5-3d179946557b\") "
Sep 12 02:04:51.020950 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.020864    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brbns\" (UniqueName: \"kubernetes.io/projected/2e36106c-a86c-47e3-9ad5-3d179946557b-kube-api-access-brbns\") pod \"2e36106c-a86c-47e3-9ad5-3d179946557b\" (UID: \"2e36106c-a86c-47e3-9ad5-3d179946557b\") "
Sep 12 02:04:51.021151 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.021124    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e36106c-a86c-47e3-9ad5-3d179946557b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e36106c-a86c-47e3-9ad5-3d179946557b" (UID: "2e36106c-a86c-47e3-9ad5-3d179946557b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:04:51.022893 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.022864    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e36106c-a86c-47e3-9ad5-3d179946557b-kube-api-access-brbns" (OuterVolumeSpecName: "kube-api-access-brbns") pod "2e36106c-a86c-47e3-9ad5-3d179946557b" (UID: "2e36106c-a86c-47e3-9ad5-3d179946557b"). InnerVolumeSpecName "kube-api-access-brbns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:04:51.121999 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.121967    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e36106c-a86c-47e3-9ad5-3d179946557b-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:04:51.121999 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.121995    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brbns\" (UniqueName: \"kubernetes.io/projected/2e36106c-a86c-47e3-9ad5-3d179946557b-kube-api-access-brbns\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:04:51.257566 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.257529    2580 generic.go:356] "Generic (PLEG): container finished" podID="2e36106c-a86c-47e3-9ad5-3d179946557b" containerID="399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38" exitCode=0
Sep 12 02:04:51.257734 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.257572    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"
Sep 12 02:04:51.257734 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.257612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s" event={"ID":"2e36106c-a86c-47e3-9ad5-3d179946557b","Type":"ContainerDied","Data":"399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38"}
Sep 12 02:04:51.257734 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.257666    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s" event={"ID":"2e36106c-a86c-47e3-9ad5-3d179946557b","Type":"ContainerDied","Data":"0023473077f2f46f7ae2bc56cbb28fbc7bd02bdfa9ceaf0891917ea31d7bbb9d"}
Sep 12 02:04:51.257734 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.257691    2580 scope.go:117] "RemoveContainer" containerID="399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38"
Sep 12 02:04:51.268627 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.268605    2580 scope.go:117] "RemoveContainer" containerID="399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38"
Sep 12 02:04:51.268967 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:04:51.268934    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38\": container with ID starting with 399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38 not found: ID does not exist" containerID="399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38"
Sep 12 02:04:51.269070 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.268979    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38"} err="failed to get container status \"399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38\": rpc error: code = NotFound desc = could not find container \"399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38\": container with ID starting with 399b25fdc6b2a4be1f1f1e75ac83c72276660b3bd0e146eca983b1cd6144bd38 not found: ID does not exist"
Sep 12 02:04:51.280282 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.280256    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"]
Sep 12 02:04:51.285610 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:51.285584    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3fc028-5bfb4c674-hwb4s"]
Sep 12 02:04:52.209914 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:52.209880    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"]
Sep 12 02:04:52.980873 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:52.980778    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e36106c-a86c-47e3-9ad5-3d179946557b" path="/var/lib/kubelet/pods/2e36106c-a86c-47e3-9ad5-3d179946557b/volumes"
Sep 12 02:04:53.276052 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:53.276017    2580 generic.go:356] "Generic (PLEG): container finished" podID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerID="700714d18cd4069fb47040744cd2f9a7b3c6f0d5b2b9c965e2b4f8262ab0ebe1" exitCode=0
Sep 12 02:04:53.276440 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:53.276061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" event={"ID":"3a6bdc5c-073f-4328-8d5f-612be56daa64","Type":"ContainerDied","Data":"700714d18cd4069fb47040744cd2f9a7b3c6f0d5b2b9c965e2b4f8262ab0ebe1"}
Sep 12 02:04:53.276606 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:53.276586    2580 scope.go:117] "RemoveContainer" containerID="700714d18cd4069fb47040744cd2f9a7b3c6f0d5b2b9c965e2b4f8262ab0ebe1"
Sep 12 02:04:53.277910 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:53.277881    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:04:53.277980 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:53.277951    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:04:53.278017 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:53.277986    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:04:54.292428 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:54.292377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" event={"ID":"3a6bdc5c-073f-4328-8d5f-612be56daa64","Type":"ContainerStarted","Data":"0fd1c98905d48a92fc25e4ad537356cd96a77410dd6428f6d395455f1888beff"}
Sep 12 02:04:54.823299 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:54.823253    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:04:54.823530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:54.823507    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 12 02:04:54.823578 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:54.823561    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 12 02:04:55.059754 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:55.059723    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:04:55.064549 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:55.064524    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:04:55.153252 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:55.153146    2580 scope.go:117] "RemoveContainer" containerID="48b9a3f8cd1194c5be20c34fb220e76462d5dea6423507da5161807fc7db342c"
Sep 12 02:04:55.823708 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:55.823673    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 12 02:04:55.824098 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:55.823729    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 12 02:04:56.823148 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:56.823102    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 12 02:04:56.823373 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:56.823178    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 12 02:04:57.585238 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.581524    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"]
Sep 12 02:04:57.585238 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.582189    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e36106c-a86c-47e3-9ad5-3d179946557b" containerName="payload-pre-processing"
Sep 12 02:04:57.589236 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.588246    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e36106c-a86c-47e3-9ad5-3d179946557b" containerName="payload-pre-processing"
Sep 12 02:04:57.589236 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.588295    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3477a257-c705-4c61-85bd-aac544455bb4" containerName="revoke-keys"
Sep 12 02:04:57.589236 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.588306    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3477a257-c705-4c61-85bd-aac544455bb4" containerName="revoke-keys"
Sep 12 02:04:57.589236 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.588517    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e36106c-a86c-47e3-9ad5-3d179946557b" containerName="payload-pre-processing"
Sep 12 02:04:57.589236 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.588534    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3477a257-c705-4c61-85bd-aac544455bb4" containerName="revoke-keys"
Sep 12 02:04:57.593240 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.592034    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"
Sep 12 02:04:57.601231 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.600564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:04:57.601231 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.600840    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:04:57.601429 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.601405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"]
Sep 12 02:04:57.702894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.702859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de92bec-e2a0-4161-95f0-baf79080ae80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56\" (UID: \"2de92bec-e2a0-4161-95f0-baf79080ae80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"
Sep 12 02:04:57.803737 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.803701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de92bec-e2a0-4161-95f0-baf79080ae80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56\" (UID: \"2de92bec-e2a0-4161-95f0-baf79080ae80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"
Sep 12 02:04:57.804713 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.804677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de92bec-e2a0-4161-95f0-baf79080ae80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56\" (UID: \"2de92bec-e2a0-4161-95f0-baf79080ae80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"
Sep 12 02:04:57.823816 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.823784    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 12 02:04:57.823951 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.823853    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 12 02:04:57.915185 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:57.915084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"
Sep 12 02:04:58.211047 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:58.211018    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"]
Sep 12 02:04:58.212774 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:04:58.212717    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de92bec_e2a0_4161_95f0_baf79080ae80.slice/crio-16e5736f52c6fda1475c465db36ad405d59027790304ec30fa13cc06cd3d685e WatchSource:0}: Error finding container 16e5736f52c6fda1475c465db36ad405d59027790304ec30fa13cc06cd3d685e: Status 404 returned error can't find the container with id 16e5736f52c6fda1475c465db36ad405d59027790304ec30fa13cc06cd3d685e
Sep 12 02:04:58.322465 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:58.322420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56" event={"ID":"2de92bec-e2a0-4161-95f0-baf79080ae80","Type":"ContainerStarted","Data":"16e5736f52c6fda1475c465db36ad405d59027790304ec30fa13cc06cd3d685e"}
Sep 12 02:04:58.823866 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:58.823773    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 12 02:04:58.823866 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:58.823833    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 12 02:04:59.332160 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:59.332067    2580 generic.go:356] "Generic (PLEG): container finished" podID="2de92bec-e2a0-4161-95f0-baf79080ae80" containerID="7e955c9a1b5b2d6db89c3052ad70fe533447f8c77f749c4110de6824c3c1ebac" exitCode=0
Sep 12 02:04:59.332160 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:59.332130    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56" event={"ID":"2de92bec-e2a0-4161-95f0-baf79080ae80","Type":"ContainerDied","Data":"7e955c9a1b5b2d6db89c3052ad70fe533447f8c77f749c4110de6824c3c1ebac"}
Sep 12 02:04:59.823806 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:59.823770    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 12 02:04:59.823990 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:04:59.823828    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 12 02:05:00.486166 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:00.486140    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"
Sep 12 02:05:00.639186 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:00.639080    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de92bec-e2a0-4161-95f0-baf79080ae80-maas-api-service-ca\") pod \"2de92bec-e2a0-4161-95f0-baf79080ae80\" (UID: \"2de92bec-e2a0-4161-95f0-baf79080ae80\") "
Sep 12 02:05:00.639532 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:00.639505    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de92bec-e2a0-4161-95f0-baf79080ae80-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2de92bec-e2a0-4161-95f0-baf79080ae80" (UID: "2de92bec-e2a0-4161-95f0-baf79080ae80"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:05:00.741062 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:00.741032    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de92bec-e2a0-4161-95f0-baf79080ae80-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:00.827826 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:00.827796    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:05:00.828278 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:00.828061    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:05:00.828866 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:00.828846    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:05:01.347851 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:01.347822    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"
Sep 12 02:05:01.348037 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:01.347863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56" event={"ID":"2de92bec-e2a0-4161-95f0-baf79080ae80","Type":"ContainerDied","Data":"16e5736f52c6fda1475c465db36ad405d59027790304ec30fa13cc06cd3d685e"}
Sep 12 02:05:01.348037 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:01.347908    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16e5736f52c6fda1475c465db36ad405d59027790304ec30fa13cc06cd3d685e"
Sep 12 02:05:07.229432 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.229365    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw" podUID="ff7bc83b-6cb9-49db-add4-16bbb2a719ef" containerName="main" containerID="cri-o://9cbf08bb0229e5f65b137c4a45251d4d78889daf25153b7d98168a6276f31b03" gracePeriod=45
Sep 12 02:05:07.391901 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.391858    2580 generic.go:356] "Generic (PLEG): container finished" podID="ff7bc83b-6cb9-49db-add4-16bbb2a719ef" containerID="9cbf08bb0229e5f65b137c4a45251d4d78889daf25153b7d98168a6276f31b03" exitCode=0
Sep 12 02:05:07.392044 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.391935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw" event={"ID":"ff7bc83b-6cb9-49db-add4-16bbb2a719ef","Type":"ContainerDied","Data":"9cbf08bb0229e5f65b137c4a45251d4d78889daf25153b7d98168a6276f31b03"}
Sep 12 02:05:07.486541 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.486473    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:05:07.610648 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.610615    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tmp-dir\") pod \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") "
Sep 12 02:05:07.610818 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.610658    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-dshm\") pod \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") "
Sep 12 02:05:07.610818 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.610721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tls-certs\") pod \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") "
Sep 12 02:05:07.610818 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.610766    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhn4s\" (UniqueName: \"kubernetes.io/projected/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-kube-api-access-zhn4s\") pod \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") "
Sep 12 02:05:07.610996 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.610903    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff7bc83b-6cb9-49db-add4-16bbb2a719ef" (UID: "ff7bc83b-6cb9-49db-add4-16bbb2a719ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:05:07.610996 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.610926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-home\") pod \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") "
Sep 12 02:05:07.610996 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.610962    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-model-cache\") pod \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\" (UID: \"ff7bc83b-6cb9-49db-add4-16bbb2a719ef\") "
Sep 12 02:05:07.611246 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.611190    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-home" (OuterVolumeSpecName: "home") pod "ff7bc83b-6cb9-49db-add4-16bbb2a719ef" (UID: "ff7bc83b-6cb9-49db-add4-16bbb2a719ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:05:07.611333 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.611277    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff7bc83b-6cb9-49db-add4-16bbb2a719ef" (UID: "ff7bc83b-6cb9-49db-add4-16bbb2a719ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:05:07.611445 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.611427    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:07.611501 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.611452    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:07.611501 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.611467    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:07.613017 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.612972    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff7bc83b-6cb9-49db-add4-16bbb2a719ef" (UID: "ff7bc83b-6cb9-49db-add4-16bbb2a719ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:05:07.613130 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.613015    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-dshm" (OuterVolumeSpecName: "dshm") pod "ff7bc83b-6cb9-49db-add4-16bbb2a719ef" (UID: "ff7bc83b-6cb9-49db-add4-16bbb2a719ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:05:07.613130 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.613033    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-kube-api-access-zhn4s" (OuterVolumeSpecName: "kube-api-access-zhn4s") pod "ff7bc83b-6cb9-49db-add4-16bbb2a719ef" (UID: "ff7bc83b-6cb9-49db-add4-16bbb2a719ef"). InnerVolumeSpecName "kube-api-access-zhn4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:05:07.712900 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.712868    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:07.712900 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.712895    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:07.712900 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:07.712907    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhn4s\" (UniqueName: \"kubernetes.io/projected/ff7bc83b-6cb9-49db-add4-16bbb2a719ef-kube-api-access-zhn4s\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:08.400018 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:08.399988    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"
Sep 12 02:05:08.400518 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:08.400030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw" event={"ID":"ff7bc83b-6cb9-49db-add4-16bbb2a719ef","Type":"ContainerDied","Data":"2a7162f5eba932a2b067b4f70855ffcf4db3742c7d927f10f9a528442eaff418"}
Sep 12 02:05:08.400518 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:08.400079    2580 scope.go:117] "RemoveContainer" containerID="9cbf08bb0229e5f65b137c4a45251d4d78889daf25153b7d98168a6276f31b03"
Sep 12 02:05:08.421404 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:08.421379    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"]
Sep 12 02:05:08.427252 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:08.427228    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-b99e21/noten-model-b99e21-kserve-69759ff699-pqgnw"]
Sep 12 02:05:08.980328 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:08.980285    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff7bc83b-6cb9-49db-add4-16bbb2a719ef" path="/var/lib/kubelet/pods/ff7bc83b-6cb9-49db-add4-16bbb2a719ef/volumes"
Sep 12 02:05:10.292806 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:10.292775    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs"]
Sep 12 02:05:10.295476 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:10.295447    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-7d6e9f-29819640-7n6zs"]
Sep 12 02:05:10.504792 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:10.504760    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"]
Sep 12 02:05:10.507563 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:10.507540    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6e9f-6ll56"]
Sep 12 02:05:10.981046 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:10.981013    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2de92bec-e2a0-4161-95f0-baf79080ae80" path="/var/lib/kubelet/pods/2de92bec-e2a0-4161-95f0-baf79080ae80/volumes"
Sep 12 02:05:10.981322 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:10.981309    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71ac7959-254c-4512-9917-1666ea8a04e2" path="/var/lib/kubelet/pods/71ac7959-254c-4512-9917-1666ea8a04e2/volumes"
Sep 12 02:05:22.223516 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:22.223482    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"]
Sep 12 02:05:22.224101 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:22.223759    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy" containerID="cri-o://a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709" gracePeriod=30
Sep 12 02:05:23.127721 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.127689    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"]
Sep 12 02:05:23.128332 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.128303    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7bc83b-6cb9-49db-add4-16bbb2a719ef" containerName="main"
Sep 12 02:05:23.128332 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.128327    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7bc83b-6cb9-49db-add4-16bbb2a719ef" containerName="main"
Sep 12 02:05:23.128332 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.128337    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de92bec-e2a0-4161-95f0-baf79080ae80" containerName="revoke-keys"
Sep 12 02:05:23.128587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.128345    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de92bec-e2a0-4161-95f0-baf79080ae80" containerName="revoke-keys"
Sep 12 02:05:23.128587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.128455    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de92bec-e2a0-4161-95f0-baf79080ae80" containerName="revoke-keys"
Sep 12 02:05:23.128587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.128471    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff7bc83b-6cb9-49db-add4-16bbb2a719ef" containerName="main"
Sep 12 02:05:23.133025 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.133005    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.145388 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.145362    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-a744d5-openshift-default-dockercfg-2n8kr\""
Sep 12 02:05:23.150279 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.150249    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"]
Sep 12 02:05:23.156366 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.156339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-envoy\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.156482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.156385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7njvk\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-kube-api-access-7njvk\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.156527 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.156488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-socket\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.156527 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.156504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24a215c5-6700-4b72-a14e-bd1d515d7161-istiod-ca-cert\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.156601 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.156541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-podinfo\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.156601 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.156565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-data\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.156601 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.156585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-token\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.156722 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.156700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-certs\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.156773 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.156756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-credential-socket\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.257963 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.257926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-data\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.257963 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.257968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-token\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.258574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-certs\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.258574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-credential-socket\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.258574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-envoy\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.258574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7njvk\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-kube-api-access-7njvk\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.258574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-socket\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.258574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24a215c5-6700-4b72-a14e-bd1d515d7161-istiod-ca-cert\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.258574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-podinfo\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.258574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-data\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.259026 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258659    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-certs\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.259026 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.258732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-socket\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.259324 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.259302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-credential-socket\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.259711 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.259684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24a215c5-6700-4b72-a14e-bd1d515d7161-istiod-ca-cert\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.262797 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.262763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-podinfo\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.263156 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.263101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-envoy\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.267076 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.267044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-token\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.268032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.268002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7njvk\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-kube-api-access-7njvk\") pod \"e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") " pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.443812 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.443715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:23.724028 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.723993    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"]
Sep 12 02:05:23.724650 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:05:23.724625    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a215c5_6700_4b72_a14e_bd1d515d7161.slice/crio-e27045dc1e890d32e1343bf25f8d59d4527848c2a5c0d26e0931a6a51688f99a WatchSource:0}: Error finding container e27045dc1e890d32e1343bf25f8d59d4527848c2a5c0d26e0931a6a51688f99a: Status 404 returned error can't find the container with id e27045dc1e890d32e1343bf25f8d59d4527848c2a5c0d26e0931a6a51688f99a
Sep 12 02:05:23.727080 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.727044    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:05:23.727177 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.727113    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:05:23.727177 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:23.727146    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:05:24.526323 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:24.526287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" event={"ID":"24a215c5-6700-4b72-a14e-bd1d515d7161","Type":"ContainerStarted","Data":"674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45"}
Sep 12 02:05:24.526808 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:24.526343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" event={"ID":"24a215c5-6700-4b72-a14e-bd1d515d7161","Type":"ContainerStarted","Data":"e27045dc1e890d32e1343bf25f8d59d4527848c2a5c0d26e0931a6a51688f99a"}
Sep 12 02:05:24.555066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:24.555002    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" podStartSLOduration=1.5549864310000001 podStartE2EDuration="1.554986431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:05:24.552687642 +0000 UTC m=+1230.079588721" watchObservedRunningTime="2026-09-12 02:05:24.554986431 +0000 UTC m=+1230.081887511"
Sep 12 02:05:25.444770 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:25.444728    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:25.450112 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:25.450084    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:25.533439 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:25.533409    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:25.534283 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:25.534262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:27.495557 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.495526    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 02:05:27.549344 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.549307    2580 generic.go:356] "Generic (PLEG): container finished" podID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerID="a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709" exitCode=0
Sep 12 02:05:27.549556 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.549359    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"
Sep 12 02:05:27.549556 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.549391    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" event={"ID":"b17d3b35-603b-465c-aed7-e0ee82447dda","Type":"ContainerDied","Data":"a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709"}
Sep 12 02:05:27.549556 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.549466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4" event={"ID":"b17d3b35-603b-465c-aed7-e0ee82447dda","Type":"ContainerDied","Data":"5435d35b6e587aed21f82c22e28d55964a5ac8a397b8f54b92a1e0919a337389"}
Sep 12 02:05:27.549556 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.549504    2580 scope.go:117] "RemoveContainer" containerID="a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709"
Sep 12 02:05:27.559864 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.559846    2580 scope.go:117] "RemoveContainer" containerID="d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5"
Sep 12 02:05:27.575142 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.575120    2580 scope.go:117] "RemoveContainer" containerID="a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709"
Sep 12 02:05:27.575522 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:05:27.575500    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709\": container with ID starting with a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709 not found: ID does not exist" containerID="a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709"
Sep 12 02:05:27.575594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.575534    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709"} err="failed to get container status \"a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709\": rpc error: code = NotFound desc = could not find container \"a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709\": container with ID starting with a83c17c2b5e2f671a7cd58f944dc7d819a87798f4c1c428ca28d55532a5ce709 not found: ID does not exist"
Sep 12 02:05:27.575594 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.575551    2580 scope.go:117] "RemoveContainer" containerID="d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5"
Sep 12 02:05:27.575873 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:05:27.575853    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5\": container with ID starting with d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5 not found: ID does not exist" containerID="d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5"
Sep 12 02:05:27.575941 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.575880    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5"} err="failed to get container status \"d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5\": rpc error: code = NotFound desc = could not find container \"d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5\": container with ID starting with d5a5955dc15dea62d947e87744fab45691acafb14b70431fc3da7784a7f6feb5 not found: ID does not exist"
Sep 12 02:05:27.606025 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.605993    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b17d3b35-603b-465c-aed7-e0ee82447dda-istiod-ca-cert\") pod \"b17d3b35-603b-465c-aed7-e0ee82447dda\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") "
Sep 12 02:05:27.606198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606051    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-token\") pod \"b17d3b35-603b-465c-aed7-e0ee82447dda\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") "
Sep 12 02:05:27.606198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606070    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-envoy\") pod \"b17d3b35-603b-465c-aed7-e0ee82447dda\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") "
Sep 12 02:05:27.606198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606090    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd6cw\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-kube-api-access-gd6cw\") pod \"b17d3b35-603b-465c-aed7-e0ee82447dda\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") "
Sep 12 02:05:27.606198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606145    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-credential-socket\") pod \"b17d3b35-603b-465c-aed7-e0ee82447dda\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") "
Sep 12 02:05:27.606198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-data\") pod \"b17d3b35-603b-465c-aed7-e0ee82447dda\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") "
Sep 12 02:05:27.606472 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606291    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-certs\") pod \"b17d3b35-603b-465c-aed7-e0ee82447dda\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") "
Sep 12 02:05:27.606472 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606321    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-podinfo\") pod \"b17d3b35-603b-465c-aed7-e0ee82447dda\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") "
Sep 12 02:05:27.606472 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606359    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-socket\") pod \"b17d3b35-603b-465c-aed7-e0ee82447dda\" (UID: \"b17d3b35-603b-465c-aed7-e0ee82447dda\") "
Sep 12 02:05:27.606641 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606513    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b17d3b35-603b-465c-aed7-e0ee82447dda-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b17d3b35-603b-465c-aed7-e0ee82447dda" (UID: "b17d3b35-603b-465c-aed7-e0ee82447dda"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:05:27.606699 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606506    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-data" (OuterVolumeSpecName: "istio-data") pod "b17d3b35-603b-465c-aed7-e0ee82447dda" (UID: "b17d3b35-603b-465c-aed7-e0ee82447dda"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:05:27.606699 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606563    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b17d3b35-603b-465c-aed7-e0ee82447dda" (UID: "b17d3b35-603b-465c-aed7-e0ee82447dda"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:05:27.606804 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606703    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b17d3b35-603b-465c-aed7-e0ee82447dda" (UID: "b17d3b35-603b-465c-aed7-e0ee82447dda"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:05:27.607115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606880    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:27.607115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606905    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:27.607115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606920    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:27.607115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.606933    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b17d3b35-603b-465c-aed7-e0ee82447dda-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:27.607456 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.607315    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b17d3b35-603b-465c-aed7-e0ee82447dda" (UID: "b17d3b35-603b-465c-aed7-e0ee82447dda"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:05:27.608474 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.608450    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-token" (OuterVolumeSpecName: "istio-token") pod "b17d3b35-603b-465c-aed7-e0ee82447dda" (UID: "b17d3b35-603b-465c-aed7-e0ee82447dda"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:05:27.608474 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.608467    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b17d3b35-603b-465c-aed7-e0ee82447dda" (UID: "b17d3b35-603b-465c-aed7-e0ee82447dda"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:05:27.608700 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.608679    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-kube-api-access-gd6cw" (OuterVolumeSpecName: "kube-api-access-gd6cw") pod "b17d3b35-603b-465c-aed7-e0ee82447dda" (UID: "b17d3b35-603b-465c-aed7-e0ee82447dda"). InnerVolumeSpecName "kube-api-access-gd6cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:05:27.608916 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.608888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b17d3b35-603b-465c-aed7-e0ee82447dda" (UID: "b17d3b35-603b-465c-aed7-e0ee82447dda"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:05:27.708606 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.708564    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:27.708606 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.708598    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:27.708606 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.708613    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd6cw\" (UniqueName: \"kubernetes.io/projected/b17d3b35-603b-465c-aed7-e0ee82447dda-kube-api-access-gd6cw\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:27.708862 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.708626    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b17d3b35-603b-465c-aed7-e0ee82447dda-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:27.708862 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.708641    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b17d3b35-603b-465c-aed7-e0ee82447dda-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:05:27.874528 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.874496    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"]
Sep 12 02:05:27.877161 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:27.877129    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7d6e9f-openshift-default-f485ddc98-6vsd4"]
Sep 12 02:05:28.981754 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:28.981708    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" path="/var/lib/kubelet/pods/b17d3b35-603b-465c-aed7-e0ee82447dda/volumes"
Sep 12 02:05:31.353481 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.353441    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"]
Sep 12 02:05:31.353860 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.353852    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy"
Sep 12 02:05:31.353903 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.353864    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy"
Sep 12 02:05:31.353958 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.353947    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy"
Sep 12 02:05:31.353991 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.353960    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy"
Sep 12 02:05:31.358418 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.358358    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:31.360961 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.360933    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-a744d5\""
Sep 12 02:05:31.361109 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.360974    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-a744d5-dockercfg-xbd6d\""
Sep 12 02:05:31.368675 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.368645    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"]
Sep 12 02:05:31.444986 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.444941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ce1f88-2488-4f94-b22c-51422a8dda16-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762\" (UID: \"19ce1f88-2488-4f94-b22c-51422a8dda16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:31.444986 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.444989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-676z4\" (UniqueName: \"kubernetes.io/projected/19ce1f88-2488-4f94-b22c-51422a8dda16-kube-api-access-676z4\") pod \"payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762\" (UID: \"19ce1f88-2488-4f94-b22c-51422a8dda16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:31.546696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.546655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ce1f88-2488-4f94-b22c-51422a8dda16-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762\" (UID: \"19ce1f88-2488-4f94-b22c-51422a8dda16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:31.546696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.546698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-676z4\" (UniqueName: \"kubernetes.io/projected/19ce1f88-2488-4f94-b22c-51422a8dda16-kube-api-access-676z4\") pod \"payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762\" (UID: \"19ce1f88-2488-4f94-b22c-51422a8dda16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:31.547433 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.547412    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ce1f88-2488-4f94-b22c-51422a8dda16-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762\" (UID: \"19ce1f88-2488-4f94-b22c-51422a8dda16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:31.554515 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.554492    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-676z4\" (UniqueName: \"kubernetes.io/projected/19ce1f88-2488-4f94-b22c-51422a8dda16-kube-api-access-676z4\") pod \"payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762\" (UID: \"19ce1f88-2488-4f94-b22c-51422a8dda16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:31.673268 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.673167    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:31.961878 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:31.961853    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"]
Sep 12 02:05:31.964235 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:05:31.964189    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ce1f88_2488_4f94_b22c_51422a8dda16.slice/crio-53f059a1a448ccd6a2306284ebafc6b91ed673b37b4cdf1615ad975be345c2af WatchSource:0}: Error finding container 53f059a1a448ccd6a2306284ebafc6b91ed673b37b4cdf1615ad975be345c2af: Status 404 returned error can't find the container with id 53f059a1a448ccd6a2306284ebafc6b91ed673b37b4cdf1615ad975be345c2af
Sep 12 02:05:32.587541 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:32.587504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762" event={"ID":"19ce1f88-2488-4f94-b22c-51422a8dda16","Type":"ContainerStarted","Data":"27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580"}
Sep 12 02:05:32.588019 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:32.587563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762" event={"ID":"19ce1f88-2488-4f94-b22c-51422a8dda16","Type":"ContainerStarted","Data":"53f059a1a448ccd6a2306284ebafc6b91ed673b37b4cdf1615ad975be345c2af"}
Sep 12 02:05:32.588019 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:32.587691    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:32.606439 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:32.606382    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762" podStartSLOduration=1.606365583 podStartE2EDuration="1.606365583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:05:32.603983807 +0000 UTC m=+1238.130884888" watchObservedRunningTime="2026-09-12 02:05:32.606365583 +0000 UTC m=+1238.133266720"
Sep 12 02:05:43.597225 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:43.597179    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:05:50.714016 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:50.713978    2580 generic.go:356] "Generic (PLEG): container finished" podID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerID="674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45" exitCode=0
Sep 12 02:05:50.714463 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:50.714053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" event={"ID":"24a215c5-6700-4b72-a14e-bd1d515d7161","Type":"ContainerDied","Data":"674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45"}
Sep 12 02:05:50.714533 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:50.714520    2580 scope.go:117] "RemoveContainer" containerID="674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45"
Sep 12 02:05:50.715661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:50.715637    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:05:50.715712 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:50.715700    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:05:50.715757 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:50.715729    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:05:51.723296 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:51.723257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" event={"ID":"24a215c5-6700-4b72-a14e-bd1d515d7161","Type":"ContainerStarted","Data":"9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db"}
Sep 12 02:05:52.444542 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:52.444500    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:52.445768 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:52.445746    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 12 02:05:52.445829 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:52.445813    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 12 02:05:53.444831 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:53.444788    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 12 02:05:53.445262 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:53.444850    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 12 02:05:54.444955 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:54.444913    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 12 02:05:54.445379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:54.444975    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 12 02:05:55.204322 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:55.204278    2580 scope.go:117] "RemoveContainer" containerID="791b6d960402d0d41bac3607fa209fa3e7910dcedd9c2c9e804f55b247708251"
Sep 12 02:05:55.444670 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:55.444632    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 12 02:05:55.444918 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:55.444696    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 12 02:05:55.534197 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:55.534164    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:56.444258 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:56.444202    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 12 02:05:56.444437 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:56.444278    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 12 02:05:57.448709 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:57.448680    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:05:57.449677 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:05:57.449656    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:06:05.097883 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.097849    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"]
Sep 12 02:06:05.098483 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.098461    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy"
Sep 12 02:06:05.098573 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.098484    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b17d3b35-603b-465c-aed7-e0ee82447dda" containerName="istio-proxy"
Sep 12 02:06:05.103438 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.103413    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:05.106015 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.105993    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-fbb6d9\""
Sep 12 02:06:05.106147 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.106042    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-fbb6d9-dockercfg-cslvm\""
Sep 12 02:06:05.112956 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.112918    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"]
Sep 12 02:06:05.218703 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.218667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfac920a-6df2-443c-af07-8cecee6b4d84-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr\" (UID: \"dfac920a-6df2-443c-af07-8cecee6b4d84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:05.218883 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.218845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79stg\" (UniqueName: \"kubernetes.io/projected/dfac920a-6df2-443c-af07-8cecee6b4d84-kube-api-access-79stg\") pod \"payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr\" (UID: \"dfac920a-6df2-443c-af07-8cecee6b4d84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:05.320101 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.320063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79stg\" (UniqueName: \"kubernetes.io/projected/dfac920a-6df2-443c-af07-8cecee6b4d84-kube-api-access-79stg\") pod \"payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr\" (UID: \"dfac920a-6df2-443c-af07-8cecee6b4d84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:05.320338 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.320130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfac920a-6df2-443c-af07-8cecee6b4d84-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr\" (UID: \"dfac920a-6df2-443c-af07-8cecee6b4d84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:05.320884 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.320862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfac920a-6df2-443c-af07-8cecee6b4d84-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr\" (UID: \"dfac920a-6df2-443c-af07-8cecee6b4d84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:05.337271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.337237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79stg\" (UniqueName: \"kubernetes.io/projected/dfac920a-6df2-443c-af07-8cecee6b4d84-kube-api-access-79stg\") pod \"payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr\" (UID: \"dfac920a-6df2-443c-af07-8cecee6b4d84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:05.417160 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.417061    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:05.703307 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.703282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"]
Sep 12 02:06:05.705284 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:06:05.705255    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfac920a_6df2_443c_af07_8cecee6b4d84.slice/crio-7e660722bb1df2744b36867ba5c8a3bba3a3fb85e58d0e46b163cb6fd32011e4 WatchSource:0}: Error finding container 7e660722bb1df2744b36867ba5c8a3bba3a3fb85e58d0e46b163cb6fd32011e4: Status 404 returned error can't find the container with id 7e660722bb1df2744b36867ba5c8a3bba3a3fb85e58d0e46b163cb6fd32011e4
Sep 12 02:06:05.819083 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.819050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr" event={"ID":"dfac920a-6df2-443c-af07-8cecee6b4d84","Type":"ContainerStarted","Data":"7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee"}
Sep 12 02:06:05.819269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.819101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr" event={"ID":"dfac920a-6df2-443c-af07-8cecee6b4d84","Type":"ContainerStarted","Data":"7e660722bb1df2744b36867ba5c8a3bba3a3fb85e58d0e46b163cb6fd32011e4"}
Sep 12 02:06:05.819269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.819134    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:05.836427 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:05.836364    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr" podStartSLOduration=0.836342605 podStartE2EDuration="836.342605ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:06:05.834135704 +0000 UTC m=+1271.361036788" watchObservedRunningTime="2026-09-12 02:06:05.836342605 +0000 UTC m=+1271.363243686"
Sep 12 02:06:16.828405 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:16.828372    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:06:25.171242 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:25.171190    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"]
Sep 12 02:06:40.191977 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.191928    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn" podUID="db65a202-818d-441e-abba-aa801362caf4" containerName="main" containerID="cri-o://d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644" gracePeriod=45
Sep 12 02:06:40.441611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.441584    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:06:40.492599 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.492508    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db65a202-818d-441e-abba-aa801362caf4-tls-certs\") pod \"db65a202-818d-441e-abba-aa801362caf4\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") "
Sep 12 02:06:40.492599 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.492552    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-model-cache\") pod \"db65a202-818d-441e-abba-aa801362caf4\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") "
Sep 12 02:06:40.492599 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.492574    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-tmp-dir\") pod \"db65a202-818d-441e-abba-aa801362caf4\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") "
Sep 12 02:06:40.492894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.492635    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qkzt\" (UniqueName: \"kubernetes.io/projected/db65a202-818d-441e-abba-aa801362caf4-kube-api-access-2qkzt\") pod \"db65a202-818d-441e-abba-aa801362caf4\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") "
Sep 12 02:06:40.492894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.492682    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-home\") pod \"db65a202-818d-441e-abba-aa801362caf4\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") "
Sep 12 02:06:40.492894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.492733    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-dshm\") pod \"db65a202-818d-441e-abba-aa801362caf4\" (UID: \"db65a202-818d-441e-abba-aa801362caf4\") "
Sep 12 02:06:40.492894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.492818    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-model-cache" (OuterVolumeSpecName: "model-cache") pod "db65a202-818d-441e-abba-aa801362caf4" (UID: "db65a202-818d-441e-abba-aa801362caf4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:06:40.492894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.492881    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db65a202-818d-441e-abba-aa801362caf4" (UID: "db65a202-818d-441e-abba-aa801362caf4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:06:40.493144 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.493028    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-home" (OuterVolumeSpecName: "home") pod "db65a202-818d-441e-abba-aa801362caf4" (UID: "db65a202-818d-441e-abba-aa801362caf4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:06:40.493144 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.493130    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:06:40.493271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.493147    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:06:40.493271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.493161    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:06:40.495177 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.495151    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-dshm" (OuterVolumeSpecName: "dshm") pod "db65a202-818d-441e-abba-aa801362caf4" (UID: "db65a202-818d-441e-abba-aa801362caf4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:06:40.495306 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.495268    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db65a202-818d-441e-abba-aa801362caf4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db65a202-818d-441e-abba-aa801362caf4" (UID: "db65a202-818d-441e-abba-aa801362caf4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:06:40.495306 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.495294    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db65a202-818d-441e-abba-aa801362caf4-kube-api-access-2qkzt" (OuterVolumeSpecName: "kube-api-access-2qkzt") pod "db65a202-818d-441e-abba-aa801362caf4" (UID: "db65a202-818d-441e-abba-aa801362caf4"). InnerVolumeSpecName "kube-api-access-2qkzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:06:40.593833 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.593793    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qkzt\" (UniqueName: \"kubernetes.io/projected/db65a202-818d-441e-abba-aa801362caf4-kube-api-access-2qkzt\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:06:40.593833 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.593826    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db65a202-818d-441e-abba-aa801362caf4-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:06:40.593833 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:40.593836    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db65a202-818d-441e-abba-aa801362caf4-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:06:41.046586 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:41.046552    2580 generic.go:356] "Generic (PLEG): container finished" podID="db65a202-818d-441e-abba-aa801362caf4" containerID="d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644" exitCode=0
Sep 12 02:06:41.046586 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:41.046592    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"
Sep 12 02:06:41.046797 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:41.046637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn" event={"ID":"db65a202-818d-441e-abba-aa801362caf4","Type":"ContainerDied","Data":"d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644"}
Sep 12 02:06:41.046797 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:41.046678    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn" event={"ID":"db65a202-818d-441e-abba-aa801362caf4","Type":"ContainerDied","Data":"b75a4e5e8b17cf7fd357fdd0ae143b11e038db7ee72ace9538e06fd6d0e835eb"}
Sep 12 02:06:41.046797 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:41.046700    2580 scope.go:117] "RemoveContainer" containerID="d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644"
Sep 12 02:06:41.056663 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:41.056645    2580 scope.go:117] "RemoveContainer" containerID="d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644"
Sep 12 02:06:41.056964 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:06:41.056940    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644\": container with ID starting with d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644 not found: ID does not exist" containerID="d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644"
Sep 12 02:06:41.057012 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:41.056975    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644"} err="failed to get container status \"d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644\": rpc error: code = NotFound desc = could not find container \"d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644\": container with ID starting with d24f75e4b4c0f94cc5144f736a84d1ea8a7d0ac9b30d893fc90f2c8af93fa644 not found: ID does not exist"
Sep 12 02:06:41.064914 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:41.064889    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"]
Sep 12 02:06:41.073707 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:41.073679    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/e2e-conflict-a-3901f64e-kserve-6f6f6968d5-zjzcn"]
Sep 12 02:06:42.980629 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:42.980594    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db65a202-818d-441e-abba-aa801362caf4" path="/var/lib/kubelet/pods/db65a202-818d-441e-abba-aa801362caf4/volumes"
Sep 12 02:06:49.231162 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.231126    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"]
Sep 12 02:06:49.231597 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.231582    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db65a202-818d-441e-abba-aa801362caf4" containerName="main"
Sep 12 02:06:49.231597 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.231598    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db65a202-818d-441e-abba-aa801362caf4" containerName="main"
Sep 12 02:06:49.231678 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.231672    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db65a202-818d-441e-abba-aa801362caf4" containerName="main"
Sep 12 02:06:49.234966 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.234946    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.237612 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.237587    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-98f68e\"/\"distinct-w4-98f68e-kserve-self-signed-certs\""
Sep 12 02:06:49.243195 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.243164    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"]
Sep 12 02:06:49.386532 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.386498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e140dcef-3920-4126-b647-689a29b66540-tls-certs\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.386820 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.386793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-tmp-dir\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.386965 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.386863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-model-cache\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.386965 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.386951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-dshm\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.387091 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.387006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzjgb\" (UniqueName: \"kubernetes.io/projected/e140dcef-3920-4126-b647-689a29b66540-kube-api-access-bzjgb\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.387091 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.387061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-home\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.488701 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.488621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzjgb\" (UniqueName: \"kubernetes.io/projected/e140dcef-3920-4126-b647-689a29b66540-kube-api-access-bzjgb\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.488701 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.488674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-home\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.488902 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.488717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e140dcef-3920-4126-b647-689a29b66540-tls-certs\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.488902 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.488745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-tmp-dir\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.488902 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.488788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-model-cache\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.488902 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.488830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-dshm\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.489312 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.489280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-model-cache\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.489408 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.489306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-home\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.489408 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.489349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-tmp-dir\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.491042 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.491025    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-dshm\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.491448 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.491430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e140dcef-3920-4126-b647-689a29b66540-tls-certs\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.497352 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.497327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzjgb\" (UniqueName: \"kubernetes.io/projected/e140dcef-3920-4126-b647-689a29b66540-kube-api-access-bzjgb\") pod \"distinct-w4-98f68e-kserve-554b58fc7-btbng\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") " pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.545083 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.545052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:49.819978 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:49.819947    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"]
Sep 12 02:06:49.820262 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:06:49.820238    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode140dcef_3920_4126_b647_689a29b66540.slice/crio-7d345ad2f5b075403268666f86fb5c5ee397647b7ffc99e129eb2bfcd02ed6ec WatchSource:0}: Error finding container 7d345ad2f5b075403268666f86fb5c5ee397647b7ffc99e129eb2bfcd02ed6ec: Status 404 returned error can't find the container with id 7d345ad2f5b075403268666f86fb5c5ee397647b7ffc99e129eb2bfcd02ed6ec
Sep 12 02:06:50.114459 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:50.114375    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng" event={"ID":"e140dcef-3920-4126-b647-689a29b66540","Type":"ContainerStarted","Data":"3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4"}
Sep 12 02:06:50.114459 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:50.114419    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng" event={"ID":"e140dcef-3920-4126-b647-689a29b66540","Type":"ContainerStarted","Data":"7d345ad2f5b075403268666f86fb5c5ee397647b7ffc99e129eb2bfcd02ed6ec"}
Sep 12 02:06:50.132019 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:50.131967    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng" podStartSLOduration=1.131950186 podStartE2EDuration="1.131950186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:06:50.129604703 +0000 UTC m=+1315.656505784" watchObservedRunningTime="2026-09-12 02:06:50.131950186 +0000 UTC m=+1315.658851270"
Sep 12 02:06:55.241238 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:55.241193    2580 scope.go:117] "RemoveContainer" containerID="d85665af8eedcdb4d6315fe9bdaf617d0429c4662f13dfce5db6bdd18e5a8bb7"
Sep 12 02:06:55.251266 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:55.251246    2580 scope.go:117] "RemoveContainer" containerID="0a9c9f97a7e2eaeaac9b325c360552b233fcb9de54aa8300f36020bf64ff0ff8"
Sep 12 02:06:55.265104 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:55.265073    2580 scope.go:117] "RemoveContainer" containerID="10dd848eb2811bc4d55548ccbf11addf97ffb602a4dbcfed5feca762cb4176e9"
Sep 12 02:06:55.280973 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:55.280947    2580 scope.go:117] "RemoveContainer" containerID="48b11e9318a1cd8a95fbbdbba995eb9e4c56c64ff1218b12535c2d03333d195e"
Sep 12 02:06:55.293978 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:55.293952    2580 scope.go:117] "RemoveContainer" containerID="7aee1a8b0e89dac67e9eb496a4d507e10bf583b55e0795b16e93694e74072129"
Sep 12 02:06:55.303118 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:55.303094    2580 scope.go:117] "RemoveContainer" containerID="bbe84386087496f8e7981d1e273d0e55993a8d328dfb60e83ce6084aa494e6f7"
Sep 12 02:06:57.636431 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.636394    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"]
Sep 12 02:06:57.642323 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.642303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.646363 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.646332    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-a744d5\"/\"openshift-service-ca.crt\""
Sep 12 02:06:57.646495 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.646409    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a744d5\"/\"auth-test-model-a744d5-kserve-self-signed-certs\""
Sep 12 02:06:57.646495 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.646412    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-a744d5\"/\"kube-root-ca.crt\""
Sep 12 02:06:57.646495 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.646418    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a744d5\"/\"default-dockercfg-fb9kj\""
Sep 12 02:06:57.654502 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.654475    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"]
Sep 12 02:06:57.685282 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.685248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-dshm\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.685451 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.685309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-tmp-dir\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.685451 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.685427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-model-cache\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.685591 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.685454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0e05f23-6882-409e-9db3-bf61ceac8a03-tls-certs\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.685591 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.685564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m76fj\" (UniqueName: \"kubernetes.io/projected/c0e05f23-6882-409e-9db3-bf61ceac8a03-kube-api-access-m76fj\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.685710 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.685595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-home\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.786319 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.786284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-tmp-dir\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.786504 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.786354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-model-cache\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.786504 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.786470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0e05f23-6882-409e-9db3-bf61ceac8a03-tls-certs\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.786646 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.786542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m76fj\" (UniqueName: \"kubernetes.io/projected/c0e05f23-6882-409e-9db3-bf61ceac8a03-kube-api-access-m76fj\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.786646 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.786570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-home\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.786646 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.786619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-dshm\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.786807 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.786731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-model-cache\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.786807 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.786777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-tmp-dir\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.786971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.786949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-home\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.788733 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.788703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-dshm\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.789020 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.788998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0e05f23-6882-409e-9db3-bf61ceac8a03-tls-certs\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.794957 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.794937    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m76fj\" (UniqueName: \"kubernetes.io/projected/c0e05f23-6882-409e-9db3-bf61ceac8a03-kube-api-access-m76fj\") pod \"auth-test-model-a744d5-kserve-66bb4cff87-hlrkp\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:57.953314 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:57.953199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:06:58.232907 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:58.232865    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"]
Sep 12 02:06:58.233559 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:06:58.233532    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0e05f23_6882_409e_9db3_bf61ceac8a03.slice/crio-76329b5b5080a44eee736a78fb5434397f1e7906022c1beb71642984fc7508ab WatchSource:0}: Error finding container 76329b5b5080a44eee736a78fb5434397f1e7906022c1beb71642984fc7508ab: Status 404 returned error can't find the container with id 76329b5b5080a44eee736a78fb5434397f1e7906022c1beb71642984fc7508ab
Sep 12 02:06:59.209907 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:59.209863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp" event={"ID":"c0e05f23-6882-409e-9db3-bf61ceac8a03","Type":"ContainerStarted","Data":"c7b53dbfc6b0313b7e220fdcc22edeb2eca306a39275f0f9a3396ae8868e8ba0"}
Sep 12 02:06:59.210313 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:59.209913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp" event={"ID":"c0e05f23-6882-409e-9db3-bf61ceac8a03","Type":"ContainerStarted","Data":"76329b5b5080a44eee736a78fb5434397f1e7906022c1beb71642984fc7508ab"}
Sep 12 02:06:59.227409 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:59.227360    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp" podStartSLOduration=2.227342795 podStartE2EDuration="2.227342795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:06:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:06:59.225608544 +0000 UTC m=+1324.752509626" watchObservedRunningTime="2026-09-12 02:06:59.227342795 +0000 UTC m=+1324.754243874"
Sep 12 02:06:59.545480 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:59.545438    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:59.545676 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:59.545529    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:06:59.557898 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:06:59.557869    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:07:00.227394 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:00.227367    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:07:07.954283 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:07.954246    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:07:07.954697 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:07.954499    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:07:07.967060 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:07.967038    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:07:08.284618 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:08.284589    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:07:15.528940 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:15.528903    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"]
Sep 12 02:07:15.532863 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:15.532846    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"
Sep 12 02:07:15.535329 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:15.535310    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:07:15.535443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:15.535310    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:07:15.540704 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:15.540680    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"]
Sep 12 02:07:15.676465 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:15.676422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/139619a7-89c3-491e-875e-618e39d59e29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-60c2473b-ffcls\" (UID: \"139619a7-89c3-491e-875e-618e39d59e29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"
Sep 12 02:07:15.777954 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:15.777912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/139619a7-89c3-491e-875e-618e39d59e29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-60c2473b-ffcls\" (UID: \"139619a7-89c3-491e-875e-618e39d59e29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"
Sep 12 02:07:15.778630 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:15.778604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/139619a7-89c3-491e-875e-618e39d59e29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-60c2473b-ffcls\" (UID: \"139619a7-89c3-491e-875e-618e39d59e29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"
Sep 12 02:07:15.845855 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:15.845766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"
Sep 12 02:07:16.121728 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:16.121696    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"]
Sep 12 02:07:16.122974 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:07:16.122948    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139619a7_89c3_491e_875e_618e39d59e29.slice/crio-c2edd748e1123bdebdff8938f3ce21cc6ca6db057838f23fb247fdcc96a315cf WatchSource:0}: Error finding container c2edd748e1123bdebdff8938f3ce21cc6ca6db057838f23fb247fdcc96a315cf: Status 404 returned error can't find the container with id c2edd748e1123bdebdff8938f3ce21cc6ca6db057838f23fb247fdcc96a315cf
Sep 12 02:07:16.324545 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:16.324509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls" event={"ID":"139619a7-89c3-491e-875e-618e39d59e29","Type":"ContainerStarted","Data":"c2edd748e1123bdebdff8938f3ce21cc6ca6db057838f23fb247fdcc96a315cf"}
Sep 12 02:07:17.334333 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:17.334244    2580 generic.go:356] "Generic (PLEG): container finished" podID="139619a7-89c3-491e-875e-618e39d59e29" containerID="5b0c39561c09f28f239f3a04c601b4a3c4d5769171758022e7f810190a352eed" exitCode=0
Sep 12 02:07:17.335002 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:17.334970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls" event={"ID":"139619a7-89c3-491e-875e-618e39d59e29","Type":"ContainerDied","Data":"5b0c39561c09f28f239f3a04c601b4a3c4d5769171758022e7f810190a352eed"}
Sep 12 02:07:18.486743 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:18.486714    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"
Sep 12 02:07:18.607816 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:18.607714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/139619a7-89c3-491e-875e-618e39d59e29-maas-api-service-ca\") pod \"139619a7-89c3-491e-875e-618e39d59e29\" (UID: \"139619a7-89c3-491e-875e-618e39d59e29\") "
Sep 12 02:07:18.608078 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:18.608053    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139619a7-89c3-491e-875e-618e39d59e29-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "139619a7-89c3-491e-875e-618e39d59e29" (UID: "139619a7-89c3-491e-875e-618e39d59e29"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:07:18.709179 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:18.709147    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/139619a7-89c3-491e-875e-618e39d59e29-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:07:19.348576 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:19.348542    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"
Sep 12 02:07:19.348576 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:19.348548    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls" event={"ID":"139619a7-89c3-491e-875e-618e39d59e29","Type":"ContainerDied","Data":"c2edd748e1123bdebdff8938f3ce21cc6ca6db057838f23fb247fdcc96a315cf"}
Sep 12 02:07:19.348799 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:19.348595    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2edd748e1123bdebdff8938f3ce21cc6ca6db057838f23fb247fdcc96a315cf"
Sep 12 02:07:31.254487 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:31.254445    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"]
Sep 12 02:07:31.257262 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:31.257234    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-60c2473b-ffcls"]
Sep 12 02:07:32.980282 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:07:32.980244    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="139619a7-89c3-491e-875e-618e39d59e29" path="/var/lib/kubelet/pods/139619a7-89c3-491e-875e-618e39d59e29/volumes"
Sep 12 02:09:55.117891 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:55.117859    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:09:55.130622 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:55.130597    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:09:55.395493 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:55.395402    2580 scope.go:117] "RemoveContainer" containerID="c9e20595245b4c5cd2a9098d320319708f5dfbe7192e50da919e0e5b672a6556"
Sep 12 02:09:55.404968 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:55.404940    2580 scope.go:117] "RemoveContainer" containerID="5bb4f89d140d7d1e41aa1c95c5e9d0e23d2cd217c84964dcc4062a1c790bc237"
Sep 12 02:09:59.669529 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.669480    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"]
Sep 12 02:09:59.670143 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.670113    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="139619a7-89c3-491e-875e-618e39d59e29" containerName="revoke-keys"
Sep 12 02:09:59.670143 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.670144    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="139619a7-89c3-491e-875e-618e39d59e29" containerName="revoke-keys"
Sep 12 02:09:59.670328 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.670317    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="139619a7-89c3-491e-875e-618e39d59e29" containerName="revoke-keys"
Sep 12 02:09:59.676399 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.676369    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"
Sep 12 02:09:59.679309 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.679272    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:09:59.679539 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.679521    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:09:59.680273 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.680251    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"]
Sep 12 02:09:59.731683 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.731636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89f210f-363a-4807-a39d-629276fc86aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p\" (UID: \"d89f210f-363a-4807-a39d-629276fc86aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"
Sep 12 02:09:59.833321 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.833269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89f210f-363a-4807-a39d-629276fc86aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p\" (UID: \"d89f210f-363a-4807-a39d-629276fc86aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"
Sep 12 02:09:59.833929 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.833910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89f210f-363a-4807-a39d-629276fc86aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p\" (UID: \"d89f210f-363a-4807-a39d-629276fc86aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"
Sep 12 02:09:59.991268 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:09:59.991172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"
Sep 12 02:10:00.265295 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:00.265268    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"]
Sep 12 02:10:00.266884 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:10:00.266857    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89f210f_363a_4807_a39d_629276fc86aa.slice/crio-a4747d7e001ca575ea514a641f797120f11012208ca77448b830b69e44c7fa50 WatchSource:0}: Error finding container a4747d7e001ca575ea514a641f797120f11012208ca77448b830b69e44c7fa50: Status 404 returned error can't find the container with id a4747d7e001ca575ea514a641f797120f11012208ca77448b830b69e44c7fa50
Sep 12 02:10:00.383975 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:00.383932    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p" event={"ID":"d89f210f-363a-4807-a39d-629276fc86aa","Type":"ContainerStarted","Data":"a4747d7e001ca575ea514a641f797120f11012208ca77448b830b69e44c7fa50"}
Sep 12 02:10:01.391944 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:01.391841    2580 generic.go:356] "Generic (PLEG): container finished" podID="d89f210f-363a-4807-a39d-629276fc86aa" containerID="5e5eea3b724535eb8e9ab6de64f3870078dd878d001e502b61892fe57b66a78e" exitCode=0
Sep 12 02:10:01.391944 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:01.391927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p" event={"ID":"d89f210f-363a-4807-a39d-629276fc86aa","Type":"ContainerDied","Data":"5e5eea3b724535eb8e9ab6de64f3870078dd878d001e502b61892fe57b66a78e"}
Sep 12 02:10:02.543801 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:02.543776    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"
Sep 12 02:10:02.664580 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:02.664489    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89f210f-363a-4807-a39d-629276fc86aa-maas-api-service-ca\") pod \"d89f210f-363a-4807-a39d-629276fc86aa\" (UID: \"d89f210f-363a-4807-a39d-629276fc86aa\") "
Sep 12 02:10:02.664847 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:02.664824    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d89f210f-363a-4807-a39d-629276fc86aa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d89f210f-363a-4807-a39d-629276fc86aa" (UID: "d89f210f-363a-4807-a39d-629276fc86aa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:10:02.765662 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:02.765627    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89f210f-363a-4807-a39d-629276fc86aa-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:03.406331 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:03.406298    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"
Sep 12 02:10:03.406511 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:03.406344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p" event={"ID":"d89f210f-363a-4807-a39d-629276fc86aa","Type":"ContainerDied","Data":"a4747d7e001ca575ea514a641f797120f11012208ca77448b830b69e44c7fa50"}
Sep 12 02:10:03.406511 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:03.406399    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4747d7e001ca575ea514a641f797120f11012208ca77448b830b69e44c7fa50"
Sep 12 02:10:04.611017 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:04.610983    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"]
Sep 12 02:10:04.629643 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:04.629607    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"]
Sep 12 02:10:05.295261 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.295201    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"]
Sep 12 02:10:05.295659 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.295646    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89f210f-363a-4807-a39d-629276fc86aa" containerName="revoke-keys"
Sep 12 02:10:05.295715 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.295661    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89f210f-363a-4807-a39d-629276fc86aa" containerName="revoke-keys"
Sep 12 02:10:05.295752 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.295744    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d89f210f-363a-4807-a39d-629276fc86aa" containerName="revoke-keys"
Sep 12 02:10:05.299133 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.299111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.301698 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.301672    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b23bee-openshift-default-dockercfg-47t8s\""
Sep 12 02:10:05.312470 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.312442    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"]
Sep 12 02:10:05.394906 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.394863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5963f3-dec5-4108-afa0-f9244d6ae488-istiod-ca-cert\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.395123 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.394956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-credential-socket\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.395123 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.394995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-data\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.395123 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.395099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-socket\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.395338 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.395205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-token\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.395338 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.395273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-podinfo\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.395338 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.395321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-certs\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.395503 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.395376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54k7r\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-kube-api-access-54k7r\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.395503 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.395418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-envoy\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496333 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-token\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496333 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-podinfo\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496576 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-certs\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496576 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54k7r\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-kube-api-access-54k7r\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496576 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-envoy\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496576 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5963f3-dec5-4108-afa0-f9244d6ae488-istiod-ca-cert\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496576 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-credential-socket\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496576 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-data\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496918 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-socket\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.496918 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-certs\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.497041 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.496956    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-socket\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.497096 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.497059    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-credential-socket\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.497155 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.497118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-data\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.497253 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.497237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5963f3-dec5-4108-afa0-f9244d6ae488-istiod-ca-cert\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.498706 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.498689    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-envoy\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.498790 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.498772    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-podinfo\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.518365 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.518332    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-token\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.518498 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.518409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54k7r\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-kube-api-access-54k7r\") pod \"e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") " pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.610672 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.610584    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:05.901706 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.901678    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"]
Sep 12 02:10:05.913769 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.913733    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:10:05.913879 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.913801    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:10:05.913879 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:05.913832    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:10:06.429945 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:06.429908    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" event={"ID":"5b5963f3-dec5-4108-afa0-f9244d6ae488","Type":"ContainerStarted","Data":"f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7"}
Sep 12 02:10:06.430128 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:06.429994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" event={"ID":"5b5963f3-dec5-4108-afa0-f9244d6ae488","Type":"ContainerStarted","Data":"d721faba10305c966a909add4ea74fc1db0c2ebf2a3b2b8d13e3f76722339e38"}
Sep 12 02:10:06.451753 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:06.451693    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" podStartSLOduration=1.451678651 podStartE2EDuration="1.451678651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:10:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:10:06.449099313 +0000 UTC m=+1511.976000395" watchObservedRunningTime="2026-09-12 02:10:06.451678651 +0000 UTC m=+1511.978579732"
Sep 12 02:10:06.610943 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:06.610905    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:06.616841 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:06.616810    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:07.437014 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:07.436975    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:07.437959 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:07.437941    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:10:18.646146 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:18.646069    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"]
Sep 12 02:10:18.651105 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:18.651076    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f660a6-m5q9p"]
Sep 12 02:10:18.981113 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:18.981039    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d89f210f-363a-4807-a39d-629276fc86aa" path="/var/lib/kubelet/pods/d89f210f-363a-4807-a39d-629276fc86aa/volumes"
Sep 12 02:10:19.631428 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.631367    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq" podUID="93407ba5-d5a5-49c5-8ad4-371a44c1d45c" containerName="main" containerID="cri-o://51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84" gracePeriod=45
Sep 12 02:10:19.649322 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.649292    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn" podUID="f8dad09b-8fca-4f6c-8339-d81618662fc3" containerName="main" containerID="cri-o://3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba" gracePeriod=45
Sep 12 02:10:19.917596 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.917568    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:10:19.942536 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.942510    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:10:19.956717 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.956682    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-model-cache\") pod \"f8dad09b-8fca-4f6c-8339-d81618662fc3\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") "
Sep 12 02:10:19.956860 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.956748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg2m9\" (UniqueName: \"kubernetes.io/projected/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-kube-api-access-tg2m9\") pod \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") "
Sep 12 02:10:19.956860 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.956771    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-dshm\") pod \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") "
Sep 12 02:10:19.956860 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.956816    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tls-certs\") pod \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") "
Sep 12 02:10:19.957043 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.956862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-model-cache\") pod \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") "
Sep 12 02:10:19.957043 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.956925    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4p2x\" (UniqueName: \"kubernetes.io/projected/f8dad09b-8fca-4f6c-8339-d81618662fc3-kube-api-access-v4p2x\") pod \"f8dad09b-8fca-4f6c-8339-d81618662fc3\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") "
Sep 12 02:10:19.957043 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.956962    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-model-cache" (OuterVolumeSpecName: "model-cache") pod "f8dad09b-8fca-4f6c-8339-d81618662fc3" (UID: "f8dad09b-8fca-4f6c-8339-d81618662fc3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:10:19.957043 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.956987    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8dad09b-8fca-4f6c-8339-d81618662fc3-tls-certs\") pod \"f8dad09b-8fca-4f6c-8339-d81618662fc3\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") "
Sep 12 02:10:19.957043 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957015    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-home\") pod \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") "
Sep 12 02:10:19.957043 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957041    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-tmp-dir\") pod \"f8dad09b-8fca-4f6c-8339-d81618662fc3\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") "
Sep 12 02:10:19.957379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957069    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tmp-dir\") pod \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\" (UID: \"93407ba5-d5a5-49c5-8ad4-371a44c1d45c\") "
Sep 12 02:10:19.957379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957105    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-home\") pod \"f8dad09b-8fca-4f6c-8339-d81618662fc3\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") "
Sep 12 02:10:19.957379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957148    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-dshm\") pod \"f8dad09b-8fca-4f6c-8339-d81618662fc3\" (UID: \"f8dad09b-8fca-4f6c-8339-d81618662fc3\") "
Sep 12 02:10:19.957379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957322    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-home" (OuterVolumeSpecName: "home") pod "93407ba5-d5a5-49c5-8ad4-371a44c1d45c" (UID: "93407ba5-d5a5-49c5-8ad4-371a44c1d45c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:10:19.957718 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957696    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "93407ba5-d5a5-49c5-8ad4-371a44c1d45c" (UID: "93407ba5-d5a5-49c5-8ad4-371a44c1d45c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:10:19.957931 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957839    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:19.957931 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957866    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:19.957931 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957881    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:19.957931 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.957891    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-model-cache" (OuterVolumeSpecName: "model-cache") pod "93407ba5-d5a5-49c5-8ad4-371a44c1d45c" (UID: "93407ba5-d5a5-49c5-8ad4-371a44c1d45c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:10:19.958227 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.958044    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f8dad09b-8fca-4f6c-8339-d81618662fc3" (UID: "f8dad09b-8fca-4f6c-8339-d81618662fc3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:10:19.958227 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.958131    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-home" (OuterVolumeSpecName: "home") pod "f8dad09b-8fca-4f6c-8339-d81618662fc3" (UID: "f8dad09b-8fca-4f6c-8339-d81618662fc3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:10:19.959924 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.959858    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8dad09b-8fca-4f6c-8339-d81618662fc3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f8dad09b-8fca-4f6c-8339-d81618662fc3" (UID: "f8dad09b-8fca-4f6c-8339-d81618662fc3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:10:19.960053 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.960024    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-dshm" (OuterVolumeSpecName: "dshm") pod "93407ba5-d5a5-49c5-8ad4-371a44c1d45c" (UID: "93407ba5-d5a5-49c5-8ad4-371a44c1d45c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:10:19.960479 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.960447    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "93407ba5-d5a5-49c5-8ad4-371a44c1d45c" (UID: "93407ba5-d5a5-49c5-8ad4-371a44c1d45c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:10:19.961126 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.961101    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-dshm" (OuterVolumeSpecName: "dshm") pod "f8dad09b-8fca-4f6c-8339-d81618662fc3" (UID: "f8dad09b-8fca-4f6c-8339-d81618662fc3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:10:19.961379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.961350    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-kube-api-access-tg2m9" (OuterVolumeSpecName: "kube-api-access-tg2m9") pod "93407ba5-d5a5-49c5-8ad4-371a44c1d45c" (UID: "93407ba5-d5a5-49c5-8ad4-371a44c1d45c"). InnerVolumeSpecName "kube-api-access-tg2m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:10:19.962383 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:19.962358    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8dad09b-8fca-4f6c-8339-d81618662fc3-kube-api-access-v4p2x" (OuterVolumeSpecName: "kube-api-access-v4p2x") pod "f8dad09b-8fca-4f6c-8339-d81618662fc3" (UID: "f8dad09b-8fca-4f6c-8339-d81618662fc3"). InnerVolumeSpecName "kube-api-access-v4p2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:10:20.058476 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.058443    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8dad09b-8fca-4f6c-8339-d81618662fc3-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:20.058476 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.058473    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:20.058684 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.058486    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:20.058684 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.058498    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8dad09b-8fca-4f6c-8339-d81618662fc3-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:20.058684 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.058510    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg2m9\" (UniqueName: \"kubernetes.io/projected/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-kube-api-access-tg2m9\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:20.058684 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.058524    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:20.058684 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.058535    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:20.058684 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.058547    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93407ba5-d5a5-49c5-8ad4-371a44c1d45c-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:20.058684 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.058561    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4p2x\" (UniqueName: \"kubernetes.io/projected/f8dad09b-8fca-4f6c-8339-d81618662fc3-kube-api-access-v4p2x\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:20.529201 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.529167    2580 generic.go:356] "Generic (PLEG): container finished" podID="f8dad09b-8fca-4f6c-8339-d81618662fc3" containerID="3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba" exitCode=0
Sep 12 02:10:20.529464 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.529205    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"
Sep 12 02:10:20.529464 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.529257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn" event={"ID":"f8dad09b-8fca-4f6c-8339-d81618662fc3","Type":"ContainerDied","Data":"3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba"}
Sep 12 02:10:20.529464 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.529298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn" event={"ID":"f8dad09b-8fca-4f6c-8339-d81618662fc3","Type":"ContainerDied","Data":"a62d9929bde1e880f5f3b4a5892d52a889a6156785b995d3421a632d6a8c7a19"}
Sep 12 02:10:20.529464 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.529325    2580 scope.go:117] "RemoveContainer" containerID="3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba"
Sep 12 02:10:20.530715 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.530690    2580 generic.go:356] "Generic (PLEG): container finished" podID="93407ba5-d5a5-49c5-8ad4-371a44c1d45c" containerID="51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84" exitCode=0
Sep 12 02:10:20.530809 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.530728    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"
Sep 12 02:10:20.530872 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.530810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq" event={"ID":"93407ba5-d5a5-49c5-8ad4-371a44c1d45c","Type":"ContainerDied","Data":"51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84"}
Sep 12 02:10:20.530872 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.530846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq" event={"ID":"93407ba5-d5a5-49c5-8ad4-371a44c1d45c","Type":"ContainerDied","Data":"aee3570ef01ee3bd4baeef98d06540718715cac52f1fa8e3e8973247f8085c69"}
Sep 12 02:10:20.539802 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.539720    2580 scope.go:117] "RemoveContainer" containerID="3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba"
Sep 12 02:10:20.540096 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:10:20.540075    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba\": container with ID starting with 3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba not found: ID does not exist" containerID="3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba"
Sep 12 02:10:20.540173 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.540105    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba"} err="failed to get container status \"3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba\": rpc error: code = NotFound desc = could not find container \"3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba\": container with ID starting with 3cacf0bd79d0d7f638fa92600b26d196c25f95088bd0b9a1a42ac75a13985eba not found: ID does not exist"
Sep 12 02:10:20.540173 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.540122    2580 scope.go:117] "RemoveContainer" containerID="51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84"
Sep 12 02:10:20.548312 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.548280    2580 scope.go:117] "RemoveContainer" containerID="51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84"
Sep 12 02:10:20.548697 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:10:20.548664    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84\": container with ID starting with 51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84 not found: ID does not exist" containerID="51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84"
Sep 12 02:10:20.548770 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.548695    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84"} err="failed to get container status \"51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84\": rpc error: code = NotFound desc = could not find container \"51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84\": container with ID starting with 51292a81b4aa0cdf4974ce70c19b86f5d36c3a540ddd5610647dfa6677522f84 not found: ID does not exist"
Sep 12 02:10:20.554566 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.554545    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"]
Sep 12 02:10:20.558748 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.558727    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/premium-w0-f660a6-kserve-58f9d867bc-27bnn"]
Sep 12 02:10:20.571255 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.571235    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"]
Sep 12 02:10:20.576904 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.576884    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f660a6/distinct-w0-f660a6-kserve-58646b5978-6d4rq"]
Sep 12 02:10:20.988499 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.988416    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93407ba5-d5a5-49c5-8ad4-371a44c1d45c" path="/var/lib/kubelet/pods/93407ba5-d5a5-49c5-8ad4-371a44c1d45c/volumes"
Sep 12 02:10:20.988836 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:20.988821    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8dad09b-8fca-4f6c-8339-d81618662fc3" path="/var/lib/kubelet/pods/f8dad09b-8fca-4f6c-8339-d81618662fc3/volumes"
Sep 12 02:10:21.540277 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:21.540242    2580 generic.go:356] "Generic (PLEG): container finished" podID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" containerID="90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9" exitCode=137
Sep 12 02:10:21.540452 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:21.540295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc" event={"ID":"5f915fcb-6af0-4e0e-9294-deb5f7c772dd","Type":"ContainerDied","Data":"90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9"}
Sep 12 02:10:21.540648 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:21.540634    2580 scope.go:117] "RemoveContainer" containerID="90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9"
Sep 12 02:10:22.548803 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:22.548770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc" event={"ID":"5f915fcb-6af0-4e0e-9294-deb5f7c772dd","Type":"ContainerStarted","Data":"6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992"}
Sep 12 02:10:22.549233 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:22.548979    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 02:10:31.197859 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.197818    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"]
Sep 12 02:10:31.198462 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.198440    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8dad09b-8fca-4f6c-8339-d81618662fc3" containerName="main"
Sep 12 02:10:31.198559 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.198464    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8dad09b-8fca-4f6c-8339-d81618662fc3" containerName="main"
Sep 12 02:10:31.198559 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.198476    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93407ba5-d5a5-49c5-8ad4-371a44c1d45c" containerName="main"
Sep 12 02:10:31.198559 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.198486    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="93407ba5-d5a5-49c5-8ad4-371a44c1d45c" containerName="main"
Sep 12 02:10:31.198723 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.198601    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8dad09b-8fca-4f6c-8339-d81618662fc3" containerName="main"
Sep 12 02:10:31.198723 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.198620    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="93407ba5-d5a5-49c5-8ad4-371a44c1d45c" containerName="main"
Sep 12 02:10:31.201900 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.201877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"
Sep 12 02:10:31.204193 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.204171    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:10:31.204322 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.204248    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:10:31.210262 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.209983    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"]
Sep 12 02:10:31.265388 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.265358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06173d44-6d68-4b27-a74f-7c2333db1615-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-caecee-wnvj8\" (UID: \"06173d44-6d68-4b27-a74f-7c2333db1615\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"
Sep 12 02:10:31.367012 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.366911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06173d44-6d68-4b27-a74f-7c2333db1615-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-caecee-wnvj8\" (UID: \"06173d44-6d68-4b27-a74f-7c2333db1615\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"
Sep 12 02:10:31.367635 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.367608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06173d44-6d68-4b27-a74f-7c2333db1615-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-caecee-wnvj8\" (UID: \"06173d44-6d68-4b27-a74f-7c2333db1615\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"
Sep 12 02:10:31.515297 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.515267    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"
Sep 12 02:10:31.787447 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:31.787410    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"]
Sep 12 02:10:31.788126 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:10:31.788104    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06173d44_6d68_4b27_a74f_7c2333db1615.slice/crio-bad34cb1b25aca141f70fea7eb00aa5a077bf09d142d021a388d39d9c338a1b7 WatchSource:0}: Error finding container bad34cb1b25aca141f70fea7eb00aa5a077bf09d142d021a388d39d9c338a1b7: Status 404 returned error can't find the container with id bad34cb1b25aca141f70fea7eb00aa5a077bf09d142d021a388d39d9c338a1b7
Sep 12 02:10:32.618511 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:32.618476    2580 generic.go:356] "Generic (PLEG): container finished" podID="06173d44-6d68-4b27-a74f-7c2333db1615" containerID="774b79c2c53c54fb5c386437a6e3c37ce115dc614d63ec21a638fa3c941c6b4c" exitCode=0
Sep 12 02:10:32.618863 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:32.618556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8" event={"ID":"06173d44-6d68-4b27-a74f-7c2333db1615","Type":"ContainerDied","Data":"774b79c2c53c54fb5c386437a6e3c37ce115dc614d63ec21a638fa3c941c6b4c"}
Sep 12 02:10:32.618863 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:32.618606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8" event={"ID":"06173d44-6d68-4b27-a74f-7c2333db1615","Type":"ContainerStarted","Data":"bad34cb1b25aca141f70fea7eb00aa5a077bf09d142d021a388d39d9c338a1b7"}
Sep 12 02:10:33.558151 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:33.558110    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 02:10:33.768792 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:33.768765    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"
Sep 12 02:10:33.892634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:33.892549    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06173d44-6d68-4b27-a74f-7c2333db1615-maas-api-service-ca\") pod \"06173d44-6d68-4b27-a74f-7c2333db1615\" (UID: \"06173d44-6d68-4b27-a74f-7c2333db1615\") "
Sep 12 02:10:33.892902 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:33.892879    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06173d44-6d68-4b27-a74f-7c2333db1615-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06173d44-6d68-4b27-a74f-7c2333db1615" (UID: "06173d44-6d68-4b27-a74f-7c2333db1615"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:10:33.993583 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:33.993552    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06173d44-6d68-4b27-a74f-7c2333db1615-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:34.635681 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:34.635652    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"
Sep 12 02:10:34.635846 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:34.635699    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8" event={"ID":"06173d44-6d68-4b27-a74f-7c2333db1615","Type":"ContainerDied","Data":"bad34cb1b25aca141f70fea7eb00aa5a077bf09d142d021a388d39d9c338a1b7"}
Sep 12 02:10:34.635846 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:34.635750    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bad34cb1b25aca141f70fea7eb00aa5a077bf09d142d021a388d39d9c338a1b7"
Sep 12 02:10:47.245875 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:47.245841    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"]
Sep 12 02:10:47.249551 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:47.249526    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-caecee-wnvj8"]
Sep 12 02:10:48.980104 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:48.980071    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06173d44-6d68-4b27-a74f-7c2333db1615" path="/var/lib/kubelet/pods/06173d44-6d68-4b27-a74f-7c2333db1615/volumes"
Sep 12 02:10:54.963175 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:54.963138    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"]
Sep 12 02:10:54.965799 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:54.963583    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06173d44-6d68-4b27-a74f-7c2333db1615" containerName="revoke-keys"
Sep 12 02:10:54.965799 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:54.963595    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="06173d44-6d68-4b27-a74f-7c2333db1615" containerName="revoke-keys"
Sep 12 02:10:54.965799 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:54.963668    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="06173d44-6d68-4b27-a74f-7c2333db1615" containerName="revoke-keys"
Sep 12 02:10:54.966755 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:54.966734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"
Sep 12 02:10:54.972233 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:54.969436    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:10:54.972233 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:54.969710    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:10:54.976727 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:54.976700    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"]
Sep 12 02:10:55.008888 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:55.008862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d2be668-0588-4c41-95e7-889f7807d1bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b23bee-q626g\" (UID: \"5d2be668-0588-4c41-95e7-889f7807d1bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"
Sep 12 02:10:55.111004 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:55.110970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d2be668-0588-4c41-95e7-889f7807d1bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b23bee-q626g\" (UID: \"5d2be668-0588-4c41-95e7-889f7807d1bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"
Sep 12 02:10:55.111663 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:55.111639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d2be668-0588-4c41-95e7-889f7807d1bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b23bee-q626g\" (UID: \"5d2be668-0588-4c41-95e7-889f7807d1bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"
Sep 12 02:10:55.296228 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:55.296178    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"
Sep 12 02:10:55.443030 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:55.439154    2580 scope.go:117] "RemoveContainer" containerID="6f435e850d4866de389028fac4cbe789b4a718309c4c4942928822ea75a91370"
Sep 12 02:10:55.594279 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:10:55.593977    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d2be668_0588_4c41_95e7_889f7807d1bd.slice/crio-85e6cf8e7fe3bc82b202f99ccab4a2001bde38d9d6598993c76a9f63a3ba9d65 WatchSource:0}: Error finding container 85e6cf8e7fe3bc82b202f99ccab4a2001bde38d9d6598993c76a9f63a3ba9d65: Status 404 returned error can't find the container with id 85e6cf8e7fe3bc82b202f99ccab4a2001bde38d9d6598993c76a9f63a3ba9d65
Sep 12 02:10:55.598436 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:55.595606    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"]
Sep 12 02:10:55.796246 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:55.796191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g" event={"ID":"5d2be668-0588-4c41-95e7-889f7807d1bd","Type":"ContainerStarted","Data":"85e6cf8e7fe3bc82b202f99ccab4a2001bde38d9d6598993c76a9f63a3ba9d65"}
Sep 12 02:10:56.804640 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:56.804602    2580 generic.go:356] "Generic (PLEG): container finished" podID="5d2be668-0588-4c41-95e7-889f7807d1bd" containerID="1dbcf17164ed31c79d3d53c6d110cb3fbac0375168fdc66b7efa7a5d00a07733" exitCode=0
Sep 12 02:10:56.805040 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:56.804674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g" event={"ID":"5d2be668-0588-4c41-95e7-889f7807d1bd","Type":"ContainerDied","Data":"1dbcf17164ed31c79d3d53c6d110cb3fbac0375168fdc66b7efa7a5d00a07733"}
Sep 12 02:10:57.963332 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:57.963308    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"
Sep 12 02:10:58.043205 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:58.043172    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d2be668-0588-4c41-95e7-889f7807d1bd-maas-api-service-ca\") pod \"5d2be668-0588-4c41-95e7-889f7807d1bd\" (UID: \"5d2be668-0588-4c41-95e7-889f7807d1bd\") "
Sep 12 02:10:58.043508 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:58.043487    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d2be668-0588-4c41-95e7-889f7807d1bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d2be668-0588-4c41-95e7-889f7807d1bd" (UID: "5d2be668-0588-4c41-95e7-889f7807d1bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:10:58.145082 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:58.145004    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d2be668-0588-4c41-95e7-889f7807d1bd-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:10:58.819077 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:58.819048    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"
Sep 12 02:10:58.819276 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:58.819113    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g" event={"ID":"5d2be668-0588-4c41-95e7-889f7807d1bd","Type":"ContainerDied","Data":"85e6cf8e7fe3bc82b202f99ccab4a2001bde38d9d6598993c76a9f63a3ba9d65"}
Sep 12 02:10:58.819276 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:10:58.819189    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85e6cf8e7fe3bc82b202f99ccab4a2001bde38d9d6598993c76a9f63a3ba9d65"
Sep 12 02:11:02.847965 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:02.847930    2580 generic.go:356] "Generic (PLEG): container finished" podID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerID="f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7" exitCode=0
Sep 12 02:11:02.848497 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:02.848008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" event={"ID":"5b5963f3-dec5-4108-afa0-f9244d6ae488","Type":"ContainerDied","Data":"f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7"}
Sep 12 02:11:02.848573 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:02.848523    2580 scope.go:117] "RemoveContainer" containerID="f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7"
Sep 12 02:11:02.849881 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:02.849850    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:11:02.849954 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:02.849932    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:11:02.849990 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:02.849981    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 12 02:11:03.857174 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:03.857133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" event={"ID":"5b5963f3-dec5-4108-afa0-f9244d6ae488","Type":"ContainerStarted","Data":"57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf"}
Sep 12 02:11:04.611066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:04.611024    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:11:04.612075 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:04.612047    2580 patch_prober.go:28] interesting pod/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 12 02:11:04.612232 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:04.612108    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 12 02:11:05.611566 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:05.611529    2580 patch_prober.go:28] interesting pod/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 12 02:11:05.612164 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:05.612128    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 12 02:11:06.611678 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:06.611634    2580 patch_prober.go:28] interesting pod/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 12 02:11:06.612164 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:06.611701    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 12 02:11:07.437769 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:07.437735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:11:07.611767 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:07.611732    2580 patch_prober.go:28] interesting pod/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 12 02:11:07.612255 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:07.611799    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 12 02:11:08.612031 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:08.611989    2580 patch_prober.go:28] interesting pod/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 12 02:11:08.612461 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:08.612061    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 12 02:11:09.615479 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:09.615450    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:11:09.616319 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:09.616302    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:11:10.804441 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:10.804405    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"]
Sep 12 02:11:10.808563 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:10.808535    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23bee-q626g"]
Sep 12 02:11:10.981100 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:10.981052    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d2be668-0588-4c41-95e7-889f7807d1bd" path="/var/lib/kubelet/pods/5d2be668-0588-4c41-95e7-889f7807d1bd/volumes"
Sep 12 02:11:17.531967 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.531929    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"]
Sep 12 02:11:17.532423 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.532172    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy" containerID="cri-o://57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf" gracePeriod=30
Sep 12 02:11:17.874067 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.873973    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"]
Sep 12 02:11:17.874494 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.874478    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d2be668-0588-4c41-95e7-889f7807d1bd" containerName="revoke-keys"
Sep 12 02:11:17.874574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.874494    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d2be668-0588-4c41-95e7-889f7807d1bd" containerName="revoke-keys"
Sep 12 02:11:17.874574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.874559    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d2be668-0588-4c41-95e7-889f7807d1bd" containerName="revoke-keys"
Sep 12 02:11:17.877399 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.877372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"
Sep 12 02:11:17.879830 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.879810    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:11:17.879930 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.879814    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:11:17.888602 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.888569    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"]
Sep 12 02:11:17.949076 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:17.949045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5965954-b7cd-4c5a-8703-06cd2804277e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf\" (UID: \"f5965954-b7cd-4c5a-8703-06cd2804277e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"
Sep 12 02:11:18.049882 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:18.049839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5965954-b7cd-4c5a-8703-06cd2804277e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf\" (UID: \"f5965954-b7cd-4c5a-8703-06cd2804277e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"
Sep 12 02:11:18.050584 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:18.050555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5965954-b7cd-4c5a-8703-06cd2804277e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf\" (UID: \"f5965954-b7cd-4c5a-8703-06cd2804277e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"
Sep 12 02:11:18.196574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:18.196495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"
Sep 12 02:11:18.479922 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:18.479894    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"]
Sep 12 02:11:18.481062 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:11:18.481033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5965954_b7cd_4c5a_8703_06cd2804277e.slice/crio-d72c4472d60a63b35895164a70d3dbbd38d2b5eccd8f252e88c9e6bfd99c84b9 WatchSource:0}: Error finding container d72c4472d60a63b35895164a70d3dbbd38d2b5eccd8f252e88c9e6bfd99c84b9: Status 404 returned error can't find the container with id d72c4472d60a63b35895164a70d3dbbd38d2b5eccd8f252e88c9e6bfd99c84b9
Sep 12 02:11:18.980502 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:18.980471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf" event={"ID":"f5965954-b7cd-4c5a-8703-06cd2804277e","Type":"ContainerStarted","Data":"402ba7633f78cf92300c2eae4366a291c0f3d3e0f1b5a87208b915c827803f4e"}
Sep 12 02:11:18.980850 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:18.980522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf" event={"ID":"f5965954-b7cd-4c5a-8703-06cd2804277e","Type":"ContainerStarted","Data":"d72c4472d60a63b35895164a70d3dbbd38d2b5eccd8f252e88c9e6bfd99c84b9"}
Sep 12 02:11:18.991690 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:18.991643    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf" podStartSLOduration=1.991629232 podStartE2EDuration="1.991629232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:11:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:11:18.989511087 +0000 UTC m=+1584.516412168" watchObservedRunningTime="2026-09-12 02:11:18.991629232 +0000 UTC m=+1584.518530310"
Sep 12 02:11:19.985536 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:19.985498    2580 generic.go:356] "Generic (PLEG): container finished" podID="f5965954-b7cd-4c5a-8703-06cd2804277e" containerID="402ba7633f78cf92300c2eae4366a291c0f3d3e0f1b5a87208b915c827803f4e" exitCode=0
Sep 12 02:11:19.985948 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:19.985591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf" event={"ID":"f5965954-b7cd-4c5a-8703-06cd2804277e","Type":"ContainerDied","Data":"402ba7633f78cf92300c2eae4366a291c0f3d3e0f1b5a87208b915c827803f4e"}
Sep 12 02:11:21.142360 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:21.142327    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"
Sep 12 02:11:21.285817 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:21.285786    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5965954-b7cd-4c5a-8703-06cd2804277e-maas-api-service-ca\") pod \"f5965954-b7cd-4c5a-8703-06cd2804277e\" (UID: \"f5965954-b7cd-4c5a-8703-06cd2804277e\") "
Sep 12 02:11:21.286198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:21.286174    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5965954-b7cd-4c5a-8703-06cd2804277e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f5965954-b7cd-4c5a-8703-06cd2804277e" (UID: "f5965954-b7cd-4c5a-8703-06cd2804277e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:11:21.387202 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:21.387170    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5965954-b7cd-4c5a-8703-06cd2804277e-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:21.999534 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:21.999500    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"
Sep 12 02:11:21.999735 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:21.999548    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf" event={"ID":"f5965954-b7cd-4c5a-8703-06cd2804277e","Type":"ContainerDied","Data":"d72c4472d60a63b35895164a70d3dbbd38d2b5eccd8f252e88c9e6bfd99c84b9"}
Sep 12 02:11:21.999735 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:21.999590    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d72c4472d60a63b35895164a70d3dbbd38d2b5eccd8f252e88c9e6bfd99c84b9"
Sep 12 02:11:22.795445 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.795380    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:11:22.902870 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.902833    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-socket\") pod \"5b5963f3-dec5-4108-afa0-f9244d6ae488\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") "
Sep 12 02:11:22.903060 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.902885    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-token\") pod \"5b5963f3-dec5-4108-afa0-f9244d6ae488\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") "
Sep 12 02:11:22.903060 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.902908    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-podinfo\") pod \"5b5963f3-dec5-4108-afa0-f9244d6ae488\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") "
Sep 12 02:11:22.903060 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.902947    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5963f3-dec5-4108-afa0-f9244d6ae488-istiod-ca-cert\") pod \"5b5963f3-dec5-4108-afa0-f9244d6ae488\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") "
Sep 12 02:11:22.903060 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.902992    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-envoy\") pod \"5b5963f3-dec5-4108-afa0-f9244d6ae488\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") "
Sep 12 02:11:22.903686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903105    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-certs\") pod \"5b5963f3-dec5-4108-afa0-f9244d6ae488\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") "
Sep 12 02:11:22.903686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903144    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5b5963f3-dec5-4108-afa0-f9244d6ae488" (UID: "5b5963f3-dec5-4108-afa0-f9244d6ae488"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:22.903686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903156    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-data\") pod \"5b5963f3-dec5-4108-afa0-f9244d6ae488\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") "
Sep 12 02:11:22.903686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903305    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-credential-socket\") pod \"5b5963f3-dec5-4108-afa0-f9244d6ae488\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") "
Sep 12 02:11:22.903686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903358    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54k7r\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-kube-api-access-54k7r\") pod \"5b5963f3-dec5-4108-afa0-f9244d6ae488\" (UID: \"5b5963f3-dec5-4108-afa0-f9244d6ae488\") "
Sep 12 02:11:22.903686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903394    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-data" (OuterVolumeSpecName: "istio-data") pod "5b5963f3-dec5-4108-afa0-f9244d6ae488" (UID: "5b5963f3-dec5-4108-afa0-f9244d6ae488"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:22.903686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903487    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5b5963f3-dec5-4108-afa0-f9244d6ae488" (UID: "5b5963f3-dec5-4108-afa0-f9244d6ae488"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:22.903686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903497    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5b5963f3-dec5-4108-afa0-f9244d6ae488" (UID: "5b5963f3-dec5-4108-afa0-f9244d6ae488"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:22.903686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903526    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b5963f3-dec5-4108-afa0-f9244d6ae488-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5b5963f3-dec5-4108-afa0-f9244d6ae488" (UID: "5b5963f3-dec5-4108-afa0-f9244d6ae488"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:11:22.904093 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903807    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:22.904093 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903830    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:22.904093 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903847    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5963f3-dec5-4108-afa0-f9244d6ae488-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:22.904093 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903860    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:22.904093 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.903874    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:22.905379 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.905351    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-token" (OuterVolumeSpecName: "istio-token") pod "5b5963f3-dec5-4108-afa0-f9244d6ae488" (UID: "5b5963f3-dec5-4108-afa0-f9244d6ae488"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:22.905494 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.905396    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5b5963f3-dec5-4108-afa0-f9244d6ae488" (UID: "5b5963f3-dec5-4108-afa0-f9244d6ae488"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:11:22.905664 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.905645    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5b5963f3-dec5-4108-afa0-f9244d6ae488" (UID: "5b5963f3-dec5-4108-afa0-f9244d6ae488"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:22.905733 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:22.905668    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-kube-api-access-54k7r" (OuterVolumeSpecName: "kube-api-access-54k7r") pod "5b5963f3-dec5-4108-afa0-f9244d6ae488" (UID: "5b5963f3-dec5-4108-afa0-f9244d6ae488"). InnerVolumeSpecName "kube-api-access-54k7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:23.004598 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.004573    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:23.004598 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.004598    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54k7r\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-kube-api-access-54k7r\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:23.004749 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.004608    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:23.004749 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.004616    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5963f3-dec5-4108-afa0-f9244d6ae488-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:23.007545 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.007520    2580 generic.go:356] "Generic (PLEG): container finished" podID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerID="57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf" exitCode=0
Sep 12 02:11:23.007649 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.007578    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"
Sep 12 02:11:23.007649 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.007598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" event={"ID":"5b5963f3-dec5-4108-afa0-f9244d6ae488","Type":"ContainerDied","Data":"57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf"}
Sep 12 02:11:23.007649 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.007648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc" event={"ID":"5b5963f3-dec5-4108-afa0-f9244d6ae488","Type":"ContainerDied","Data":"d721faba10305c966a909add4ea74fc1db0c2ebf2a3b2b8d13e3f76722339e38"}
Sep 12 02:11:23.007765 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.007670    2580 scope.go:117] "RemoveContainer" containerID="57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf"
Sep 12 02:11:23.017848 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.017833    2580 scope.go:117] "RemoveContainer" containerID="f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7"
Sep 12 02:11:23.026863 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.026846    2580 scope.go:117] "RemoveContainer" containerID="57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf"
Sep 12 02:11:23.027146 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:23.027127    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf\": container with ID starting with 57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf not found: ID does not exist" containerID="57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf"
Sep 12 02:11:23.027199 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.027155    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf"} err="failed to get container status \"57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf\": rpc error: code = NotFound desc = could not find container \"57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf\": container with ID starting with 57bc7699c4546c1f909b22dc8da1a2043e31a7d685c70815f5a642faa52a0fbf not found: ID does not exist"
Sep 12 02:11:23.027199 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.027175    2580 scope.go:117] "RemoveContainer" containerID="f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7"
Sep 12 02:11:23.027510 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:23.027486    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7\": container with ID starting with f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7 not found: ID does not exist" containerID="f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7"
Sep 12 02:11:23.027578 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.027519    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7"} err="failed to get container status \"f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7\": rpc error: code = NotFound desc = could not find container \"f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7\": container with ID starting with f0a8ccdde4fdb0fec4e54cb90a18e6e2fb88a40e48e4b36276e339628d84bbd7 not found: ID does not exist"
Sep 12 02:11:23.027864 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.027825    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"]
Sep 12 02:11:23.031240 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:23.031203    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b23bee-openshift-default-7cbb6d64b8-8rdbc"]
Sep 12 02:11:24.980810 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:24.980773    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" path="/var/lib/kubelet/pods/5b5963f3-dec5-4108-afa0-f9244d6ae488/volumes"
Sep 12 02:11:26.906292 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906258    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"]
Sep 12 02:11:26.906692 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906680    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy"
Sep 12 02:11:26.906692 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906693    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy"
Sep 12 02:11:26.906769 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906705    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy"
Sep 12 02:11:26.906769 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906711    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy"
Sep 12 02:11:26.906769 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906726    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5965954-b7cd-4c5a-8703-06cd2804277e" containerName="revoke-keys"
Sep 12 02:11:26.906769 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906731    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5965954-b7cd-4c5a-8703-06cd2804277e" containerName="revoke-keys"
Sep 12 02:11:26.906887 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906801    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy"
Sep 12 02:11:26.906887 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906812    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5965954-b7cd-4c5a-8703-06cd2804277e" containerName="revoke-keys"
Sep 12 02:11:26.906887 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.906818    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5963f3-dec5-4108-afa0-f9244d6ae488" containerName="istio-proxy"
Sep 12 02:11:26.909406 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.909390    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"
Sep 12 02:11:26.911778 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.911752    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:11:26.911778 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.911778    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:11:26.917893 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.917871    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"]
Sep 12 02:11:26.942032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:26.942006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec6980a-2283-4950-aeed-0ee377df2286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm\" (UID: \"4ec6980a-2283-4950-aeed-0ee377df2286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"
Sep 12 02:11:27.043448 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:27.043410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec6980a-2283-4950-aeed-0ee377df2286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm\" (UID: \"4ec6980a-2283-4950-aeed-0ee377df2286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"
Sep 12 02:11:27.043968 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:27.043949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec6980a-2283-4950-aeed-0ee377df2286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm\" (UID: \"4ec6980a-2283-4950-aeed-0ee377df2286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"
Sep 12 02:11:27.221027 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:27.220952    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"
Sep 12 02:11:27.493083 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:27.493057    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"]
Sep 12 02:11:27.495195 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:11:27.495170    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec6980a_2283_4950_aeed_0ee377df2286.slice/crio-11cff0d98a5e87c9960918c79cc4e7ed44ef5924786fccb2d31276231f6a0254 WatchSource:0}: Error finding container 11cff0d98a5e87c9960918c79cc4e7ed44ef5924786fccb2d31276231f6a0254: Status 404 returned error can't find the container with id 11cff0d98a5e87c9960918c79cc4e7ed44ef5924786fccb2d31276231f6a0254
Sep 12 02:11:28.041492 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:28.041457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm" event={"ID":"4ec6980a-2283-4950-aeed-0ee377df2286","Type":"ContainerStarted","Data":"667dc481d8d7b72b1782770714afcd15499c78faecca821b1efd1df723c36c9c"}
Sep 12 02:11:28.041855 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:28.041516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm" event={"ID":"4ec6980a-2283-4950-aeed-0ee377df2286","Type":"ContainerStarted","Data":"11cff0d98a5e87c9960918c79cc4e7ed44ef5924786fccb2d31276231f6a0254"}
Sep 12 02:11:28.057117 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:28.057066    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm" podStartSLOduration=2.057051253 podStartE2EDuration="2.057051253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:11:28.055849073 +0000 UTC m=+1593.582750143" watchObservedRunningTime="2026-09-12 02:11:28.057051253 +0000 UTC m=+1593.583952360"
Sep 12 02:11:28.899709 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:28.899671    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"]
Sep 12 02:11:28.902322 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:28.902299    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e8046f-lcpnf"]
Sep 12 02:11:28.984017 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:28.983983    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5965954-b7cd-4c5a-8703-06cd2804277e" path="/var/lib/kubelet/pods/f5965954-b7cd-4c5a-8703-06cd2804277e/volumes"
Sep 12 02:11:29.050147 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:29.050117    2580 generic.go:356] "Generic (PLEG): container finished" podID="4ec6980a-2283-4950-aeed-0ee377df2286" containerID="667dc481d8d7b72b1782770714afcd15499c78faecca821b1efd1df723c36c9c" exitCode=0
Sep 12 02:11:29.050534 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:29.050201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm" event={"ID":"4ec6980a-2283-4950-aeed-0ee377df2286","Type":"ContainerDied","Data":"667dc481d8d7b72b1782770714afcd15499c78faecca821b1efd1df723c36c9c"}
Sep 12 02:11:30.194802 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:30.194775    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"
Sep 12 02:11:30.277563 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:30.277525    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec6980a-2283-4950-aeed-0ee377df2286-maas-api-service-ca\") pod \"4ec6980a-2283-4950-aeed-0ee377df2286\" (UID: \"4ec6980a-2283-4950-aeed-0ee377df2286\") "
Sep 12 02:11:30.277847 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:30.277823    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ec6980a-2283-4950-aeed-0ee377df2286-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ec6980a-2283-4950-aeed-0ee377df2286" (UID: "4ec6980a-2283-4950-aeed-0ee377df2286"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:11:30.378362 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:30.378270    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec6980a-2283-4950-aeed-0ee377df2286-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:31.064426 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:31.064394    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"
Sep 12 02:11:31.064583 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:31.064431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm" event={"ID":"4ec6980a-2283-4950-aeed-0ee377df2286","Type":"ContainerDied","Data":"11cff0d98a5e87c9960918c79cc4e7ed44ef5924786fccb2d31276231f6a0254"}
Sep 12 02:11:31.064583 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:31.064494    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11cff0d98a5e87c9960918c79cc4e7ed44ef5924786fccb2d31276231f6a0254"
Sep 12 02:11:31.646781 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:31.646747    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"]
Sep 12 02:11:31.653849 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:31.653820    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"]
Sep 12 02:11:31.666598 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:31.666562    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"]
Sep 12 02:11:35.622028 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.621989    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"]
Sep 12 02:11:35.622525 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.622273    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy" containerID="cri-o://d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9" gracePeriod=30
Sep 12 02:11:35.985870 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.985796    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"]
Sep 12 02:11:35.986191 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.986180    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ec6980a-2283-4950-aeed-0ee377df2286" containerName="revoke-keys"
Sep 12 02:11:35.986246 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.986193    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ec6980a-2283-4950-aeed-0ee377df2286" containerName="revoke-keys"
Sep 12 02:11:35.986305 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.986296    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ec6980a-2283-4950-aeed-0ee377df2286" containerName="revoke-keys"
Sep 12 02:11:35.989108 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.989089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"
Sep 12 02:11:35.991945 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.991923    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:11:35.992057 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.991923    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:11:35.999153 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:35.999130    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"]
Sep 12 02:11:36.035697 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:36.035671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b744cae5-13b0-474f-a162-73f4b11596cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6\" (UID: \"b744cae5-13b0-474f-a162-73f4b11596cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"
Sep 12 02:11:36.137614 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:36.137573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b744cae5-13b0-474f-a162-73f4b11596cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6\" (UID: \"b744cae5-13b0-474f-a162-73f4b11596cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"
Sep 12 02:11:36.138519 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:36.138487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b744cae5-13b0-474f-a162-73f4b11596cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6\" (UID: \"b744cae5-13b0-474f-a162-73f4b11596cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"
Sep 12 02:11:36.300861 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:36.300831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"
Sep 12 02:11:36.573282 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:36.573246    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"]
Sep 12 02:11:36.579422 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:11:36.579394    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb744cae5_13b0_474f_a162_73f4b11596cc.slice/crio-40ce47b15ad8dbae94d180fc6654bd412973b4deff009df88f308ba0ea79e529 WatchSource:0}: Error finding container 40ce47b15ad8dbae94d180fc6654bd412973b4deff009df88f308ba0ea79e529: Status 404 returned error can't find the container with id 40ce47b15ad8dbae94d180fc6654bd412973b4deff009df88f308ba0ea79e529
Sep 12 02:11:37.107723 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:37.107691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6" event={"ID":"b744cae5-13b0-474f-a162-73f4b11596cc","Type":"ContainerStarted","Data":"950114ed0619fbffccb069cf30118db51d3a656a5b8f59ff7cb59f60178f058f"}
Sep 12 02:11:37.108074 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:37.107742    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6" event={"ID":"b744cae5-13b0-474f-a162-73f4b11596cc","Type":"ContainerStarted","Data":"40ce47b15ad8dbae94d180fc6654bd412973b4deff009df88f308ba0ea79e529"}
Sep 12 02:11:37.127185 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:37.127099    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6" podStartSLOduration=2.12708567 podStartE2EDuration="2.12708567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:11:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:11:37.125336378 +0000 UTC m=+1602.652237457" watchObservedRunningTime="2026-09-12 02:11:37.12708567 +0000 UTC m=+1602.653986749"
Sep 12 02:11:38.115477 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:38.115446    2580 generic.go:356] "Generic (PLEG): container finished" podID="b744cae5-13b0-474f-a162-73f4b11596cc" containerID="950114ed0619fbffccb069cf30118db51d3a656a5b8f59ff7cb59f60178f058f" exitCode=0
Sep 12 02:11:38.115830 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:38.115538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6" event={"ID":"b744cae5-13b0-474f-a162-73f4b11596cc","Type":"ContainerDied","Data":"950114ed0619fbffccb069cf30118db51d3a656a5b8f59ff7cb59f60178f058f"}
Sep 12 02:11:39.254616 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:39.254586    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"
Sep 12 02:11:39.370723 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:39.370686    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b744cae5-13b0-474f-a162-73f4b11596cc-maas-api-service-ca\") pod \"b744cae5-13b0-474f-a162-73f4b11596cc\" (UID: \"b744cae5-13b0-474f-a162-73f4b11596cc\") "
Sep 12 02:11:39.371059 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:39.371027    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b744cae5-13b0-474f-a162-73f4b11596cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b744cae5-13b0-474f-a162-73f4b11596cc" (UID: "b744cae5-13b0-474f-a162-73f4b11596cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:11:39.471961 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:39.471875    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b744cae5-13b0-474f-a162-73f4b11596cc-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:40.130408 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.130374    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"
Sep 12 02:11:40.130408 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.130397    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6" event={"ID":"b744cae5-13b0-474f-a162-73f4b11596cc","Type":"ContainerDied","Data":"40ce47b15ad8dbae94d180fc6654bd412973b4deff009df88f308ba0ea79e529"}
Sep 12 02:11:40.130630 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.130453    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40ce47b15ad8dbae94d180fc6654bd412973b4deff009df88f308ba0ea79e529"
Sep 12 02:11:40.885693 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.885633    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:11:40.985582 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.985542    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-certs\") pod \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") "
Sep 12 02:11:40.985735 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.985593    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-credential-socket\") pod \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") "
Sep 12 02:11:40.985825 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.985764    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-token\") pod \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") "
Sep 12 02:11:40.985894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.985836    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-socket\") pod \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") "
Sep 12 02:11:40.985894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.985865    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-envoy\") pod \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") "
Sep 12 02:11:40.986001 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.985912    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-data\") pod \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") "
Sep 12 02:11:40.986060 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986005    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "58ed9705-6d07-457a-aa38-8c4e3d9a68dc" (UID: "58ed9705-6d07-457a-aa38-8c4e3d9a68dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:40.986060 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986018    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-podinfo\") pod \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") "
Sep 12 02:11:40.986060 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986034    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "58ed9705-6d07-457a-aa38-8c4e3d9a68dc" (UID: "58ed9705-6d07-457a-aa38-8c4e3d9a68dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:40.986258 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986069    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq2gh\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-kube-api-access-mq2gh\") pod \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") "
Sep 12 02:11:40.986258 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986103    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istiod-ca-cert\") pod \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\" (UID: \"58ed9705-6d07-457a-aa38-8c4e3d9a68dc\") "
Sep 12 02:11:40.986383 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986309    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "58ed9705-6d07-457a-aa38-8c4e3d9a68dc" (UID: "58ed9705-6d07-457a-aa38-8c4e3d9a68dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:40.986383 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986341    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "58ed9705-6d07-457a-aa38-8c4e3d9a68dc" (UID: "58ed9705-6d07-457a-aa38-8c4e3d9a68dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:40.986588 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986564    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:40.986688 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986589    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:40.986688 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986603    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:40.986688 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986615    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:40.986858 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.986706    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "58ed9705-6d07-457a-aa38-8c4e3d9a68dc" (UID: "58ed9705-6d07-457a-aa38-8c4e3d9a68dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:11:40.988611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.988583    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "58ed9705-6d07-457a-aa38-8c4e3d9a68dc" (UID: "58ed9705-6d07-457a-aa38-8c4e3d9a68dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:40.988744 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.988690    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "58ed9705-6d07-457a-aa38-8c4e3d9a68dc" (UID: "58ed9705-6d07-457a-aa38-8c4e3d9a68dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:40.989065 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.989038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "58ed9705-6d07-457a-aa38-8c4e3d9a68dc" (UID: "58ed9705-6d07-457a-aa38-8c4e3d9a68dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:11:40.989065 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:40.989055    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-kube-api-access-mq2gh" (OuterVolumeSpecName: "kube-api-access-mq2gh") pod "58ed9705-6d07-457a-aa38-8c4e3d9a68dc" (UID: "58ed9705-6d07-457a-aa38-8c4e3d9a68dc"). InnerVolumeSpecName "kube-api-access-mq2gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:41.087584 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.087558    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:41.087584 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.087579    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:41.087584 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.087589    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq2gh\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-kube-api-access-mq2gh\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:41.087772 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.087598    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:41.087772 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.087606    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ed9705-6d07-457a-aa38-8c4e3d9a68dc-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:41.138923 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.138836    2580 generic.go:356] "Generic (PLEG): container finished" podID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerID="d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9" exitCode=0
Sep 12 02:11:41.138923 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.138905    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"
Sep 12 02:11:41.139127 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.138917    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" event={"ID":"58ed9705-6d07-457a-aa38-8c4e3d9a68dc","Type":"ContainerDied","Data":"d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9"}
Sep 12 02:11:41.139127 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.138973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s" event={"ID":"58ed9705-6d07-457a-aa38-8c4e3d9a68dc","Type":"ContainerDied","Data":"b7319fdfd27dec8435300d12625824f61b87b6edb857069324744bd262a30dea"}
Sep 12 02:11:41.139127 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.138995    2580 scope.go:117] "RemoveContainer" containerID="d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9"
Sep 12 02:11:41.149093 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.149076    2580 scope.go:117] "RemoveContainer" containerID="e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284"
Sep 12 02:11:41.157759 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.157734    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"]
Sep 12 02:11:41.158871 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.158838    2580 scope.go:117] "RemoveContainer" containerID="d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9"
Sep 12 02:11:41.159159 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:41.159133    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9\": container with ID starting with d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9 not found: ID does not exist" containerID="d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9"
Sep 12 02:11:41.159333 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.159167    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9"} err="failed to get container status \"d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9\": rpc error: code = NotFound desc = could not find container \"d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9\": container with ID starting with d9c94c2d20705cf39d5e4ae231fce05cc8183411188fcfc9c46df338f4039cf9 not found: ID does not exist"
Sep 12 02:11:41.159333 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.159196    2580 scope.go:117] "RemoveContainer" containerID="e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284"
Sep 12 02:11:41.159632 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:41.159596    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284\": container with ID starting with e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284 not found: ID does not exist" containerID="e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284"
Sep 12 02:11:41.159734 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.159631    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284"} err="failed to get container status \"e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284\": rpc error: code = NotFound desc = could not find container \"e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284\": container with ID starting with e4e010c5a688e31cce54b2ce7c5987bafd2947f74d48e948411a1d75b19b4284 not found: ID does not exist"
Sep 12 02:11:41.162187 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:41.162164    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e8046f-openshift-default-74fd9c45bd-mgc8s"]
Sep 12 02:11:42.980042 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:42.980008    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" path="/var/lib/kubelet/pods/58ed9705-6d07-457a-aa38-8c4e3d9a68dc/volumes"
Sep 12 02:11:44.025386 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.025349    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9"]
Sep 12 02:11:44.028424 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.028401    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-98f68e-29819640-h4dq9"]
Sep 12 02:11:44.054409 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.054384    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"]
Sep 12 02:11:44.054718 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.054661    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx" podUID="91238106-7be6-44c5-881f-fe690dbc7c69" containerName="payload-processing" containerID="cri-o://2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978" gracePeriod=30
Sep 12 02:11:44.158277 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.158240    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx" podUID="91238106-7be6-44c5-881f-fe690dbc7c69" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.52:9004: connect: connection refused"
Sep 12 02:11:44.218463 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.218426    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"]
Sep 12 02:11:44.222883 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.222856    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-98f68e-5fqqm"]
Sep 12 02:11:44.330837 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.330807    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 02:11:44.420828 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.420794    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91238106-7be6-44c5-881f-fe690dbc7c69-plugins-config-volume\") pod \"91238106-7be6-44c5-881f-fe690dbc7c69\" (UID: \"91238106-7be6-44c5-881f-fe690dbc7c69\") "
Sep 12 02:11:44.421016 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.420844    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc787\" (UniqueName: \"kubernetes.io/projected/91238106-7be6-44c5-881f-fe690dbc7c69-kube-api-access-wc787\") pod \"91238106-7be6-44c5-881f-fe690dbc7c69\" (UID: \"91238106-7be6-44c5-881f-fe690dbc7c69\") "
Sep 12 02:11:44.421267 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.421204    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91238106-7be6-44c5-881f-fe690dbc7c69-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "91238106-7be6-44c5-881f-fe690dbc7c69" (UID: "91238106-7be6-44c5-881f-fe690dbc7c69"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:11:44.422953 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.422933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91238106-7be6-44c5-881f-fe690dbc7c69-kube-api-access-wc787" (OuterVolumeSpecName: "kube-api-access-wc787") pod "91238106-7be6-44c5-881f-fe690dbc7c69" (UID: "91238106-7be6-44c5-881f-fe690dbc7c69"). InnerVolumeSpecName "kube-api-access-wc787". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:44.521773 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.521736    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91238106-7be6-44c5-881f-fe690dbc7c69-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:44.521773 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.521767    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc787\" (UniqueName: \"kubernetes.io/projected/91238106-7be6-44c5-881f-fe690dbc7c69-kube-api-access-wc787\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:44.981431 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.981387    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ec6980a-2283-4950-aeed-0ee377df2286" path="/var/lib/kubelet/pods/4ec6980a-2283-4950-aeed-0ee377df2286/volumes"
Sep 12 02:11:44.981848 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:44.981824    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df3dbb6b-d87f-41e7-826d-298cfae16122" path="/var/lib/kubelet/pods/df3dbb6b-d87f-41e7-826d-298cfae16122/volumes"
Sep 12 02:11:45.168010 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:45.167972    2580 generic.go:356] "Generic (PLEG): container finished" podID="91238106-7be6-44c5-881f-fe690dbc7c69" containerID="2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978" exitCode=0
Sep 12 02:11:45.168010 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:45.168014    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"
Sep 12 02:11:45.168539 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:45.168063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx" event={"ID":"91238106-7be6-44c5-881f-fe690dbc7c69","Type":"ContainerDied","Data":"2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978"}
Sep 12 02:11:45.168539 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:45.168119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx" event={"ID":"91238106-7be6-44c5-881f-fe690dbc7c69","Type":"ContainerDied","Data":"1fb931e6016d00d8fd9064504da8c58b2987a4ed94499cf3102ed001bf2c4001"}
Sep 12 02:11:45.168539 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:45.168148    2580 scope.go:117] "RemoveContainer" containerID="2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978"
Sep 12 02:11:45.188971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:45.188940    2580 scope.go:117] "RemoveContainer" containerID="2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978"
Sep 12 02:11:45.189280 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:45.189260    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978\": container with ID starting with 2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978 not found: ID does not exist" containerID="2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978"
Sep 12 02:11:45.189354 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:45.189289    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978"} err="failed to get container status \"2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978\": rpc error: code = NotFound desc = could not find container \"2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978\": container with ID starting with 2b853cac82ae20fc4ab2894701f2cf36b0e3db32b955e7d8c97feb0612699978 not found: ID does not exist"
Sep 12 02:11:45.198469 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:45.198443    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"]
Sep 12 02:11:45.202430 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:45.202401    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-98f68e-594bdd5679-j6vqx"]
Sep 12 02:11:46.667399 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:46.667313    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng" podUID="e140dcef-3920-4126-b647-689a29b66540" containerName="main" containerID="cri-o://3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4" gracePeriod=45
Sep 12 02:11:46.676648 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:46.676596    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw" podUID="94dfebbb-6c03-4f93-be2b-2a01ff053c22" containerName="main" containerID="cri-o://12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58" gracePeriod=45
Sep 12 02:11:46.686878 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:46.686848    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp" podUID="3fb4343d-c86f-4c11-a279-1e29fec6f992" containerName="main" containerID="cri-o://ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701" gracePeriod=45
Sep 12 02:11:46.943874 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:46.943837    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:11:46.983621 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:46.983504    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91238106-7be6-44c5-881f-fe690dbc7c69" path="/var/lib/kubelet/pods/91238106-7be6-44c5-881f-fe690dbc7c69/volumes"
Sep 12 02:11:46.986892 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:46.986858    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:11:46.995815 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:46.995793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 02:11:47.048024 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.047997    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e140dcef-3920-4126-b647-689a29b66540-tls-certs\") pod \"e140dcef-3920-4126-b647-689a29b66540\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") "
Sep 12 02:11:47.048286 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048047    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-tmp-dir\") pod \"e140dcef-3920-4126-b647-689a29b66540\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") "
Sep 12 02:11:47.048286 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048080    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-home\") pod \"e140dcef-3920-4126-b647-689a29b66540\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") "
Sep 12 02:11:47.048415 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048196    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tls-certs\") pod \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") "
Sep 12 02:11:47.048473 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048452    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzjgb\" (UniqueName: \"kubernetes.io/projected/e140dcef-3920-4126-b647-689a29b66540-kube-api-access-bzjgb\") pod \"e140dcef-3920-4126-b647-689a29b66540\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") "
Sep 12 02:11:47.048540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048508    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-model-cache\") pod \"e140dcef-3920-4126-b647-689a29b66540\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") "
Sep 12 02:11:47.048605 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048553    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-dshm\") pod \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") "
Sep 12 02:11:47.048665 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048602    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tmp-dir\") pod \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") "
Sep 12 02:11:47.048665 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048629    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-home\") pod \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") "
Sep 12 02:11:47.048665 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048662    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fjr9\" (UniqueName: \"kubernetes.io/projected/94dfebbb-6c03-4f93-be2b-2a01ff053c22-kube-api-access-8fjr9\") pod \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") "
Sep 12 02:11:47.048831 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048687    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-dshm\") pod \"e140dcef-3920-4126-b647-689a29b66540\" (UID: \"e140dcef-3920-4126-b647-689a29b66540\") "
Sep 12 02:11:47.048831 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048715    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-model-cache\") pod \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\" (UID: \"94dfebbb-6c03-4f93-be2b-2a01ff053c22\") "
Sep 12 02:11:47.049613 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-home" (OuterVolumeSpecName: "home") pod "e140dcef-3920-4126-b647-689a29b66540" (UID: "e140dcef-3920-4126-b647-689a29b66540"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.049613 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.049284    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.049844 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.049784    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "94dfebbb-6c03-4f93-be2b-2a01ff053c22" (UID: "94dfebbb-6c03-4f93-be2b-2a01ff053c22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.049844 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.048362    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e140dcef-3920-4126-b647-689a29b66540" (UID: "e140dcef-3920-4126-b647-689a29b66540"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.049844 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.049025    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-home" (OuterVolumeSpecName: "home") pod "94dfebbb-6c03-4f93-be2b-2a01ff053c22" (UID: "94dfebbb-6c03-4f93-be2b-2a01ff053c22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.049844 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.049133    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-model-cache" (OuterVolumeSpecName: "model-cache") pod "e140dcef-3920-4126-b647-689a29b66540" (UID: "e140dcef-3920-4126-b647-689a29b66540"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.050077 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.050025    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-model-cache" (OuterVolumeSpecName: "model-cache") pod "94dfebbb-6c03-4f93-be2b-2a01ff053c22" (UID: "94dfebbb-6c03-4f93-be2b-2a01ff053c22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.051651 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.051621    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e140dcef-3920-4126-b647-689a29b66540-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e140dcef-3920-4126-b647-689a29b66540" (UID: "e140dcef-3920-4126-b647-689a29b66540"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:11:47.052203 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.052128    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "94dfebbb-6c03-4f93-be2b-2a01ff053c22" (UID: "94dfebbb-6c03-4f93-be2b-2a01ff053c22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:11:47.052327 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.052238    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94dfebbb-6c03-4f93-be2b-2a01ff053c22-kube-api-access-8fjr9" (OuterVolumeSpecName: "kube-api-access-8fjr9") pod "94dfebbb-6c03-4f93-be2b-2a01ff053c22" (UID: "94dfebbb-6c03-4f93-be2b-2a01ff053c22"). InnerVolumeSpecName "kube-api-access-8fjr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:47.052327 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.052255    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e140dcef-3920-4126-b647-689a29b66540-kube-api-access-bzjgb" (OuterVolumeSpecName: "kube-api-access-bzjgb") pod "e140dcef-3920-4126-b647-689a29b66540" (UID: "e140dcef-3920-4126-b647-689a29b66540"). InnerVolumeSpecName "kube-api-access-bzjgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:47.052511 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.052489    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-dshm" (OuterVolumeSpecName: "dshm") pod "e140dcef-3920-4126-b647-689a29b66540" (UID: "e140dcef-3920-4126-b647-689a29b66540"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.052595 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.052580    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-dshm" (OuterVolumeSpecName: "dshm") pod "94dfebbb-6c03-4f93-be2b-2a01ff053c22" (UID: "94dfebbb-6c03-4f93-be2b-2a01ff053c22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.150085 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150051    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwzfb\" (UniqueName: \"kubernetes.io/projected/3fb4343d-c86f-4c11-a279-1e29fec6f992-kube-api-access-zwzfb\") pod \"3fb4343d-c86f-4c11-a279-1e29fec6f992\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") "
Sep 12 02:11:47.150085 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150089    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-tmp-dir\") pod \"3fb4343d-c86f-4c11-a279-1e29fec6f992\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") "
Sep 12 02:11:47.150340 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150138    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-dshm\") pod \"3fb4343d-c86f-4c11-a279-1e29fec6f992\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") "
Sep 12 02:11:47.150340 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150199    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-model-cache\") pod \"3fb4343d-c86f-4c11-a279-1e29fec6f992\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") "
Sep 12 02:11:47.150340 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150265    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-home\") pod \"3fb4343d-c86f-4c11-a279-1e29fec6f992\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") "
Sep 12 02:11:47.150340 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150300    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb4343d-c86f-4c11-a279-1e29fec6f992-tls-certs\") pod \"3fb4343d-c86f-4c11-a279-1e29fec6f992\" (UID: \"3fb4343d-c86f-4c11-a279-1e29fec6f992\") "
Sep 12 02:11:47.150562 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150410    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3fb4343d-c86f-4c11-a279-1e29fec6f992" (UID: "3fb4343d-c86f-4c11-a279-1e29fec6f992"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.150562 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150448    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-home" (OuterVolumeSpecName: "home") pod "3fb4343d-c86f-4c11-a279-1e29fec6f992" (UID: "3fb4343d-c86f-4c11-a279-1e29fec6f992"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.150562 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150477    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-model-cache" (OuterVolumeSpecName: "model-cache") pod "3fb4343d-c86f-4c11-a279-1e29fec6f992" (UID: "3fb4343d-c86f-4c11-a279-1e29fec6f992"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.150696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150679    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzjgb\" (UniqueName: \"kubernetes.io/projected/e140dcef-3920-4126-b647-689a29b66540-kube-api-access-bzjgb\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150736 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150699    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150736 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150709    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150736 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150719    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150736 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150731    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150738    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150747    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fjr9\" (UniqueName: \"kubernetes.io/projected/94dfebbb-6c03-4f93-be2b-2a01ff053c22-kube-api-access-8fjr9\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150755    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150763    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94dfebbb-6c03-4f93-be2b-2a01ff053c22-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150771    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150778    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150786    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e140dcef-3920-4126-b647-689a29b66540-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150798    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e140dcef-3920-4126-b647-689a29b66540-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.150876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.150810    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94dfebbb-6c03-4f93-be2b-2a01ff053c22-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.152247 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.152230    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fb4343d-c86f-4c11-a279-1e29fec6f992-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3fb4343d-c86f-4c11-a279-1e29fec6f992" (UID: "3fb4343d-c86f-4c11-a279-1e29fec6f992"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:11:47.152632 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.152617    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-dshm" (OuterVolumeSpecName: "dshm") pod "3fb4343d-c86f-4c11-a279-1e29fec6f992" (UID: "3fb4343d-c86f-4c11-a279-1e29fec6f992"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:47.152674 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.152646    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb4343d-c86f-4c11-a279-1e29fec6f992-kube-api-access-zwzfb" (OuterVolumeSpecName: "kube-api-access-zwzfb") pod "3fb4343d-c86f-4c11-a279-1e29fec6f992" (UID: "3fb4343d-c86f-4c11-a279-1e29fec6f992"). InnerVolumeSpecName "kube-api-access-zwzfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:47.184067 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.183997    2580 generic.go:356] "Generic (PLEG): container finished" podID="3fb4343d-c86f-4c11-a279-1e29fec6f992" containerID="ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701" exitCode=0
Sep 12 02:11:47.184067 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.184029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp" event={"ID":"3fb4343d-c86f-4c11-a279-1e29fec6f992","Type":"ContainerDied","Data":"ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701"}
Sep 12 02:11:47.184067 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.184049    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"
Sep 12 02:11:47.184287 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.184078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp" event={"ID":"3fb4343d-c86f-4c11-a279-1e29fec6f992","Type":"ContainerDied","Data":"e275662405888cb302d0a2fea447815b40243434df1cf385db365f586adb1aee"}
Sep 12 02:11:47.184287 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.184099    2580 scope.go:117] "RemoveContainer" containerID="ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701"
Sep 12 02:11:47.185619 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.185595    2580 generic.go:356] "Generic (PLEG): container finished" podID="94dfebbb-6c03-4f93-be2b-2a01ff053c22" containerID="12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58" exitCode=0
Sep 12 02:11:47.185727 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.185636    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"
Sep 12 02:11:47.185727 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.185664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw" event={"ID":"94dfebbb-6c03-4f93-be2b-2a01ff053c22","Type":"ContainerDied","Data":"12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58"}
Sep 12 02:11:47.185727 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.185706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw" event={"ID":"94dfebbb-6c03-4f93-be2b-2a01ff053c22","Type":"ContainerDied","Data":"b102ba52c9bb2a7a39f29fb8ff2423f54919b7da7244b44d5f80d340f3ea61e8"}
Sep 12 02:11:47.187392 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.187368    2580 generic.go:356] "Generic (PLEG): container finished" podID="e140dcef-3920-4126-b647-689a29b66540" containerID="3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4" exitCode=0
Sep 12 02:11:47.187512 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.187476    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"
Sep 12 02:11:47.187581 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.187501    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng" event={"ID":"e140dcef-3920-4126-b647-689a29b66540","Type":"ContainerDied","Data":"3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4"}
Sep 12 02:11:47.187581 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.187542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng" event={"ID":"e140dcef-3920-4126-b647-689a29b66540","Type":"ContainerDied","Data":"7d345ad2f5b075403268666f86fb5c5ee397647b7ffc99e129eb2bfcd02ed6ec"}
Sep 12 02:11:47.199508 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.199489    2580 scope.go:117] "RemoveContainer" containerID="ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701"
Sep 12 02:11:47.199813 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:47.199772    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701\": container with ID starting with ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701 not found: ID does not exist" containerID="ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701"
Sep 12 02:11:47.199891 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.199827    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701"} err="failed to get container status \"ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701\": rpc error: code = NotFound desc = could not find container \"ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701\": container with ID starting with ccacf92d6ea9913c26ba071612d6f43a6afcd10322215b56ff68bf18ceb07701 not found: ID does not exist"
Sep 12 02:11:47.199891 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.199851    2580 scope.go:117] "RemoveContainer" containerID="12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58"
Sep 12 02:11:47.208568 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.208550    2580 scope.go:117] "RemoveContainer" containerID="12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58"
Sep 12 02:11:47.208839 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:47.208821    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58\": container with ID starting with 12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58 not found: ID does not exist" containerID="12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58"
Sep 12 02:11:47.208891 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.208843    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58"} err="failed to get container status \"12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58\": rpc error: code = NotFound desc = could not find container \"12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58\": container with ID starting with 12977b9814df2b8e66fe14aec8c94c9c78403794f766522e09157176a1a1ab58 not found: ID does not exist"
Sep 12 02:11:47.208891 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.208858    2580 scope.go:117] "RemoveContainer" containerID="3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4"
Sep 12 02:11:47.217271 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.217249    2580 scope.go:117] "RemoveContainer" containerID="3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4"
Sep 12 02:11:47.217572 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:47.217553    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4\": container with ID starting with 3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4 not found: ID does not exist" containerID="3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4"
Sep 12 02:11:47.217646 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.217578    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4"} err="failed to get container status \"3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4\": rpc error: code = NotFound desc = could not find container \"3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4\": container with ID starting with 3071f98b9f3b6148335c3afd5eb3e7918c305d4e69ce0d24c017cb2afc735fa4 not found: ID does not exist"
Sep 12 02:11:47.218347 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.218328    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"]
Sep 12 02:11:47.223543 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.223522    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/simulator-w4-98f68e-kserve-7dd59dd7b7-nrmtp"]
Sep 12 02:11:47.234525 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.234506    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"]
Sep 12 02:11:47.240204 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.240185    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/premium-w4-98f68e-kserve-64859f95d6-2f7rw"]
Sep 12 02:11:47.250872 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.250851    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"]
Sep 12 02:11:47.251255 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.251236    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb4343d-c86f-4c11-a279-1e29fec6f992-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.251307 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.251258    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwzfb\" (UniqueName: \"kubernetes.io/projected/3fb4343d-c86f-4c11-a279-1e29fec6f992-kube-api-access-zwzfb\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.251307 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.251267    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb4343d-c86f-4c11-a279-1e29fec6f992-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:47.254899 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.254880    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-98f68e/distinct-w4-98f68e-kserve-554b58fc7-btbng"]
Sep 12 02:11:47.676995 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.676961    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"]
Sep 12 02:11:47.679397 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:47.679376    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ac814d-rgml6"]
Sep 12 02:11:48.980242 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:48.980193    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fb4343d-c86f-4c11-a279-1e29fec6f992" path="/var/lib/kubelet/pods/3fb4343d-c86f-4c11-a279-1e29fec6f992/volumes"
Sep 12 02:11:48.980653 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:48.980621    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94dfebbb-6c03-4f93-be2b-2a01ff053c22" path="/var/lib/kubelet/pods/94dfebbb-6c03-4f93-be2b-2a01ff053c22/volumes"
Sep 12 02:11:48.980912 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:48.980899    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b744cae5-13b0-474f-a162-73f4b11596cc" path="/var/lib/kubelet/pods/b744cae5-13b0-474f-a162-73f4b11596cc/volumes"
Sep 12 02:11:48.981541 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:48.981520    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e140dcef-3920-4126-b647-689a29b66540" path="/var/lib/kubelet/pods/e140dcef-3920-4126-b647-689a29b66540/volumes"
Sep 12 02:11:50.770585 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:50.770550    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"]
Sep 12 02:11:50.771163 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:50.770758    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" containerID="cri-o://2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2" gracePeriod=30
Sep 12 02:11:54.101061 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:54.101024    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"]
Sep 12 02:11:54.101542 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:54.101258    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy" containerID="cri-o://0fd1c98905d48a92fc25e4ad537356cd96a77410dd6428f6d395455f1888beff" gracePeriod=30
Sep 12 02:11:55.516810 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:55.516780    2580 scope.go:117] "RemoveContainer" containerID="7e955c9a1b5b2d6db89c3052ad70fe533447f8c77f749c4110de6824c3c1ebac"
Sep 12 02:11:55.525661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:55.525639    2580 scope.go:117] "RemoveContainer" containerID="a2f907661400cc9392839ccfc6c46b9b51b063e7ce325fdbc7440da628f10a31"
Sep 12 02:11:55.910530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:55.910495    2580 patch_prober.go:28] interesting pod/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 12 02:11:55.910677 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:55.910562    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 12 02:11:56.028636 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.028609    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 02:11:56.138136 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138046    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-token\") pod \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") "
Sep 12 02:11:56.138136 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138088    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-envoy\") pod \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") "
Sep 12 02:11:56.138371 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138179    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-credential-socket\") pod \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") "
Sep 12 02:11:56.138371 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5x5t\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-kube-api-access-p5x5t\") pod \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") "
Sep 12 02:11:56.138371 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138292    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istiod-ca-cert\") pod \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") "
Sep 12 02:11:56.138371 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138340    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-podinfo\") pod \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") "
Sep 12 02:11:56.138371 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138370    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-certs\") pod \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") "
Sep 12 02:11:56.138656 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138396    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-socket\") pod \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") "
Sep 12 02:11:56.138656 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138423    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-data\") pod \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\" (UID: \"e1df4bc8-7411-4f58-82f4-6c19c0c66381\") "
Sep 12 02:11:56.138656 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138508    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e1df4bc8-7411-4f58-82f4-6c19c0c66381" (UID: "e1df4bc8-7411-4f58-82f4-6c19c0c66381"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:56.138813 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138732    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e1df4bc8-7411-4f58-82f4-6c19c0c66381" (UID: "e1df4bc8-7411-4f58-82f4-6c19c0c66381"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:11:56.138873 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138846    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:56.138873 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138865    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:56.139066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.138994    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e1df4bc8-7411-4f58-82f4-6c19c0c66381" (UID: "e1df4bc8-7411-4f58-82f4-6c19c0c66381"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:56.139066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.139015    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-data" (OuterVolumeSpecName: "istio-data") pod "e1df4bc8-7411-4f58-82f4-6c19c0c66381" (UID: "e1df4bc8-7411-4f58-82f4-6c19c0c66381"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:56.139339 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.139269    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e1df4bc8-7411-4f58-82f4-6c19c0c66381" (UID: "e1df4bc8-7411-4f58-82f4-6c19c0c66381"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:56.140482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.140453    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e1df4bc8-7411-4f58-82f4-6c19c0c66381" (UID: "e1df4bc8-7411-4f58-82f4-6c19c0c66381"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:56.140582 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.140522    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-token" (OuterVolumeSpecName: "istio-token") pod "e1df4bc8-7411-4f58-82f4-6c19c0c66381" (UID: "e1df4bc8-7411-4f58-82f4-6c19c0c66381"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:56.140634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.140606    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-kube-api-access-p5x5t" (OuterVolumeSpecName: "kube-api-access-p5x5t") pod "e1df4bc8-7411-4f58-82f4-6c19c0c66381" (UID: "e1df4bc8-7411-4f58-82f4-6c19c0c66381"). InnerVolumeSpecName "kube-api-access-p5x5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:56.140722 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.140701    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e1df4bc8-7411-4f58-82f4-6c19c0c66381" (UID: "e1df4bc8-7411-4f58-82f4-6c19c0c66381"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:11:56.239399 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.239359    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5x5t\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-kube-api-access-p5x5t\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:56.239399 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.239396    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:56.239399 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.239406    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:56.239641 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.239414    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:56.239641 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.239423    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:56.239641 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.239431    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:56.239641 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.239439    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1df4bc8-7411-4f58-82f4-6c19c0c66381-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:56.249131 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.249100    2580 generic.go:356] "Generic (PLEG): container finished" podID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerID="2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2" exitCode=0
Sep 12 02:11:56.249310 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.249158    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"
Sep 12 02:11:56.249310 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.249185    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" event={"ID":"e1df4bc8-7411-4f58-82f4-6c19c0c66381","Type":"ContainerDied","Data":"2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2"}
Sep 12 02:11:56.249310 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.249279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59" event={"ID":"e1df4bc8-7411-4f58-82f4-6c19c0c66381","Type":"ContainerDied","Data":"273213ab09be7e0aac31d28866e274e8933a1d3e4c08187e884d63f0f29ee209"}
Sep 12 02:11:56.249310 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.249306    2580 scope.go:117] "RemoveContainer" containerID="2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2"
Sep 12 02:11:56.260540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.260521    2580 scope.go:117] "RemoveContainer" containerID="8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079"
Sep 12 02:11:56.270192 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.269879    2580 scope.go:117] "RemoveContainer" containerID="2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2"
Sep 12 02:11:56.270334 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:56.270226    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2\": container with ID starting with 2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2 not found: ID does not exist" containerID="2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2"
Sep 12 02:11:56.270334 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.270263    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2"} err="failed to get container status \"2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2\": rpc error: code = NotFound desc = could not find container \"2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2\": container with ID starting with 2db888075ceccb718c792366bcd8ef82b52c361fffd9fd6a3ec3ab2a4b2463e2 not found: ID does not exist"
Sep 12 02:11:56.270334 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.270288    2580 scope.go:117] "RemoveContainer" containerID="8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079"
Sep 12 02:11:56.270612 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:11:56.270588    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079\": container with ID starting with 8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079 not found: ID does not exist" containerID="8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079"
Sep 12 02:11:56.270677 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.270621    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079"} err="failed to get container status \"8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079\": rpc error: code = NotFound desc = could not find container \"8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079\": container with ID starting with 8f3d544b22a488a88852aed73872b9f25b30338ede5390956d37f429d351f079 not found: ID does not exist"
Sep 12 02:11:56.273654 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.273168    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"]
Sep 12 02:11:56.275840 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.275821    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-98f68e-openshift-default-7cd956c796-jwt59"]
Sep 12 02:11:56.980118 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:56.980079    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" path="/var/lib/kubelet/pods/e1df4bc8-7411-4f58-82f4-6c19c0c66381/volumes"
Sep 12 02:11:59.274130 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.273876    2580 generic.go:356] "Generic (PLEG): container finished" podID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerID="0fd1c98905d48a92fc25e4ad537356cd96a77410dd6428f6d395455f1888beff" exitCode=0
Sep 12 02:11:59.274130 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.274022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" event={"ID":"3a6bdc5c-073f-4328-8d5f-612be56daa64","Type":"ContainerDied","Data":"0fd1c98905d48a92fc25e4ad537356cd96a77410dd6428f6d395455f1888beff"}
Sep 12 02:11:59.274130 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.274094    2580 scope.go:117] "RemoveContainer" containerID="700714d18cd4069fb47040744cd2f9a7b3c6f0d5b2b9c965e2b4f8262ab0ebe1"
Sep 12 02:11:59.364164 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.364103    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:11:59.470052 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.469677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rc59\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-kube-api-access-6rc59\") pod \"3a6bdc5c-073f-4328-8d5f-612be56daa64\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") "
Sep 12 02:11:59.470052 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.469732    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-envoy\") pod \"3a6bdc5c-073f-4328-8d5f-612be56daa64\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") "
Sep 12 02:11:59.470052 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.469762    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-podinfo\") pod \"3a6bdc5c-073f-4328-8d5f-612be56daa64\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") "
Sep 12 02:11:59.470052 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.469802    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-socket\") pod \"3a6bdc5c-073f-4328-8d5f-612be56daa64\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") "
Sep 12 02:11:59.470052 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.469854    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-token\") pod \"3a6bdc5c-073f-4328-8d5f-612be56daa64\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") "
Sep 12 02:11:59.470052 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.469884    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-data\") pod \"3a6bdc5c-073f-4328-8d5f-612be56daa64\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") "
Sep 12 02:11:59.470052 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.469955    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-certs\") pod \"3a6bdc5c-073f-4328-8d5f-612be56daa64\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") "
Sep 12 02:11:59.470052 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.469997    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6bdc5c-073f-4328-8d5f-612be56daa64-istiod-ca-cert\") pod \"3a6bdc5c-073f-4328-8d5f-612be56daa64\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") "
Sep 12 02:11:59.471746 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.470135    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-credential-socket\") pod \"3a6bdc5c-073f-4328-8d5f-612be56daa64\" (UID: \"3a6bdc5c-073f-4328-8d5f-612be56daa64\") "
Sep 12 02:11:59.471746 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.470751    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a6bdc5c-073f-4328-8d5f-612be56daa64" (UID: "3a6bdc5c-073f-4328-8d5f-612be56daa64"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:59.471746 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.471012    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a6bdc5c-073f-4328-8d5f-612be56daa64-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a6bdc5c-073f-4328-8d5f-612be56daa64" (UID: "3a6bdc5c-073f-4328-8d5f-612be56daa64"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:11:59.471746 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.471027    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a6bdc5c-073f-4328-8d5f-612be56daa64" (UID: "3a6bdc5c-073f-4328-8d5f-612be56daa64"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:59.471746 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.471292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a6bdc5c-073f-4328-8d5f-612be56daa64" (UID: "3a6bdc5c-073f-4328-8d5f-612be56daa64"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:59.472085 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.471764    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a6bdc5c-073f-4328-8d5f-612be56daa64" (UID: "3a6bdc5c-073f-4328-8d5f-612be56daa64"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:59.473046 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.472975    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-kube-api-access-6rc59" (OuterVolumeSpecName: "kube-api-access-6rc59") pod "3a6bdc5c-073f-4328-8d5f-612be56daa64" (UID: "3a6bdc5c-073f-4328-8d5f-612be56daa64"). InnerVolumeSpecName "kube-api-access-6rc59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:59.473954 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.473923    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a6bdc5c-073f-4328-8d5f-612be56daa64" (UID: "3a6bdc5c-073f-4328-8d5f-612be56daa64"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:11:59.474547 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.474523    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a6bdc5c-073f-4328-8d5f-612be56daa64" (UID: "3a6bdc5c-073f-4328-8d5f-612be56daa64"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:11:59.474680 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.474548    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a6bdc5c-073f-4328-8d5f-612be56daa64" (UID: "3a6bdc5c-073f-4328-8d5f-612be56daa64"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:11:59.571284 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.571250    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rc59\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-kube-api-access-6rc59\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:59.571284 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.571277    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:59.571284 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.571287    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:59.571509 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.571295    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:59.571509 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.571305    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:59.571509 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.571313    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:59.571509 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.571321    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:59.571509 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.571328    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6bdc5c-073f-4328-8d5f-612be56daa64-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:11:59.571509 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:11:59.571336    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6bdc5c-073f-4328-8d5f-612be56daa64-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:12:00.282132 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:00.282103    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"
Sep 12 02:12:00.282612 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:00.282136    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x" event={"ID":"3a6bdc5c-073f-4328-8d5f-612be56daa64","Type":"ContainerDied","Data":"c8fd66306ffd542c96b5ecaec40b3eaca2eebf937a6649ec83df813d21bd8b29"}
Sep 12 02:12:00.282612 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:00.282231    2580 scope.go:117] "RemoveContainer" containerID="0fd1c98905d48a92fc25e4ad537356cd96a77410dd6428f6d395455f1888beff"
Sep 12 02:12:00.311004 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:00.308306    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"]
Sep 12 02:12:00.314115 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:00.314087    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ac814d-openshift-default-c489668fd-lrx5x"]
Sep 12 02:12:00.980302 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:00.980258    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" path="/var/lib/kubelet/pods/3a6bdc5c-073f-4328-8d5f-612be56daa64/volumes"
Sep 12 02:12:07.647911 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.647874    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8066-087ad4"]
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648254    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648266    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648275    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648281    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648290    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e140dcef-3920-4126-b647-689a29b66540" containerName="main"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648296    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e140dcef-3920-4126-b647-689a29b66540" containerName="main"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648307    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648313    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648325    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91238106-7be6-44c5-881f-fe690dbc7c69" containerName="payload-processing"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648331    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="91238106-7be6-44c5-881f-fe690dbc7c69" containerName="payload-processing"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648338    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648346    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648352    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94dfebbb-6c03-4f93-be2b-2a01ff053c22" containerName="main"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648357    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="94dfebbb-6c03-4f93-be2b-2a01ff053c22" containerName="main"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648363    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648368    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648379    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648383    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648389    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b744cae5-13b0-474f-a162-73f4b11596cc" containerName="revoke-keys"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648394    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b744cae5-13b0-474f-a162-73f4b11596cc" containerName="revoke-keys"
Sep 12 02:12:07.648391 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648402    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fb4343d-c86f-4c11-a279-1e29fec6f992" containerName="main"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648407    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fb4343d-c86f-4c11-a279-1e29fec6f992" containerName="main"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648475    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648482    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="91238106-7be6-44c5-881f-fe690dbc7c69" containerName="payload-processing"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648489    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fb4343d-c86f-4c11-a279-1e29fec6f992" containerName="main"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648494    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648499    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e140dcef-3920-4126-b647-689a29b66540" containerName="main"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648506    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648511    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1df4bc8-7411-4f58-82f4-6c19c0c66381" containerName="istio-proxy"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648517    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b744cae5-13b0-474f-a162-73f4b11596cc" containerName="revoke-keys"
Sep 12 02:12:07.649032 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.648523    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="94dfebbb-6c03-4f93-be2b-2a01ff053c22" containerName="main"
Sep 12 02:12:07.652971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.652948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-087ad4"
Sep 12 02:12:07.655628 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.655601    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dng96\""
Sep 12 02:12:07.656784 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.656766    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8066-087ad4"]
Sep 12 02:12:07.745925 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.745888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8hr5\" (UniqueName: \"kubernetes.io/projected/395fdd38-01b8-4a47-9d98-1d803e128f52-kube-api-access-d8hr5\") pod \"test-curl-8066-087ad4\" (UID: \"395fdd38-01b8-4a47-9d98-1d803e128f52\") " pod="opendatahub/test-curl-8066-087ad4"
Sep 12 02:12:07.847196 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.847139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8hr5\" (UniqueName: \"kubernetes.io/projected/395fdd38-01b8-4a47-9d98-1d803e128f52-kube-api-access-d8hr5\") pod \"test-curl-8066-087ad4\" (UID: \"395fdd38-01b8-4a47-9d98-1d803e128f52\") " pod="opendatahub/test-curl-8066-087ad4"
Sep 12 02:12:07.855011 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.854982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8hr5\" (UniqueName: \"kubernetes.io/projected/395fdd38-01b8-4a47-9d98-1d803e128f52-kube-api-access-d8hr5\") pod \"test-curl-8066-087ad4\" (UID: \"395fdd38-01b8-4a47-9d98-1d803e128f52\") " pod="opendatahub/test-curl-8066-087ad4"
Sep 12 02:12:07.964904 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:07.964816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-087ad4"
Sep 12 02:12:08.236973 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:08.236945    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8066-087ad4"]
Sep 12 02:12:08.238033 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:12:08.238006    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod395fdd38_01b8_4a47_9d98_1d803e128f52.slice/crio-24c656f807764b8fd662aa1b4e8a3600e63d6c374ece93d88b930768917f97d7 WatchSource:0}: Error finding container 24c656f807764b8fd662aa1b4e8a3600e63d6c374ece93d88b930768917f97d7: Status 404 returned error can't find the container with id 24c656f807764b8fd662aa1b4e8a3600e63d6c374ece93d88b930768917f97d7
Sep 12 02:12:08.331987 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:08.331953    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8066-087ad4" event={"ID":"395fdd38-01b8-4a47-9d98-1d803e128f52","Type":"ContainerStarted","Data":"c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e"}
Sep 12 02:12:08.332151 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:08.331998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8066-087ad4" event={"ID":"395fdd38-01b8-4a47-9d98-1d803e128f52","Type":"ContainerStarted","Data":"24c656f807764b8fd662aa1b4e8a3600e63d6c374ece93d88b930768917f97d7"}
Sep 12 02:12:08.347309 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:08.347257    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8066-087ad4" podStartSLOduration=1.347240373 podStartE2EDuration="1.347240373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:12:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:12:08.346416997 +0000 UTC m=+1633.873318080" watchObservedRunningTime="2026-09-12 02:12:08.347240373 +0000 UTC m=+1633.874141447"
Sep 12 02:12:08.980147 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:08.980113    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8066-087ad4"]
Sep 12 02:12:08.984029 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:08.984001    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8066-087ad4"]
Sep 12 02:12:09.338098 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:12:09.338069    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8066-087ad4" container="test-curl-8066-087ad4"
Sep 12 02:12:10.343508 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:10.343445    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8066-087ad4" podUID="395fdd38-01b8-4a47-9d98-1d803e128f52" containerName="test-curl-8066-087ad4" containerID="cri-o://c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e" gracePeriod=2
Sep 12 02:12:10.346041 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:12:10.346014    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8066-087ad4\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="395fdd38-01b8-4a47-9d98-1d803e128f52" pod="opendatahub/test-curl-8066-087ad4"
Sep 12 02:12:12.542147 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:12.542121    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-087ad4"
Sep 12 02:12:12.598237 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:12.598131    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8hr5\" (UniqueName: \"kubernetes.io/projected/395fdd38-01b8-4a47-9d98-1d803e128f52-kube-api-access-d8hr5\") pod \"395fdd38-01b8-4a47-9d98-1d803e128f52\" (UID: \"395fdd38-01b8-4a47-9d98-1d803e128f52\") "
Sep 12 02:12:12.600269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:12.600244    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/395fdd38-01b8-4a47-9d98-1d803e128f52-kube-api-access-d8hr5" (OuterVolumeSpecName: "kube-api-access-d8hr5") pod "395fdd38-01b8-4a47-9d98-1d803e128f52" (UID: "395fdd38-01b8-4a47-9d98-1d803e128f52"). InnerVolumeSpecName "kube-api-access-d8hr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:12:12.699515 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:12.699474    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8hr5\" (UniqueName: \"kubernetes.io/projected/395fdd38-01b8-4a47-9d98-1d803e128f52-kube-api-access-d8hr5\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:12:12.979176 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:12.979100    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="395fdd38-01b8-4a47-9d98-1d803e128f52" path="/var/lib/kubelet/pods/395fdd38-01b8-4a47-9d98-1d803e128f52/volumes"
Sep 12 02:12:13.362356 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:13.362318    2580 generic.go:356] "Generic (PLEG): container finished" podID="395fdd38-01b8-4a47-9d98-1d803e128f52" containerID="c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e" exitCode=137
Sep 12 02:12:13.362562 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:13.362367    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-087ad4"
Sep 12 02:12:13.362562 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:13.362395    2580 scope.go:117] "RemoveContainer" containerID="c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e"
Sep 12 02:12:13.372970 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:13.372941    2580 scope.go:117] "RemoveContainer" containerID="c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e"
Sep 12 02:12:13.373320 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:12:13.373299    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e\": container with ID starting with c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e not found: ID does not exist" containerID="c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e"
Sep 12 02:12:13.373417 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:13.373327    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e"} err="failed to get container status \"c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e\": rpc error: code = NotFound desc = could not find container \"c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e\": container with ID starting with c5b8640edcd61a02326cee28a95eec3b6af0e0bf3dde933aac8ae0131ee81e9e not found: ID does not exist"
Sep 12 02:12:24.259883 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:24.259843    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"]
Sep 12 02:12:31.524155 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.524119    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"]
Sep 12 02:12:31.524572 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.524555    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="395fdd38-01b8-4a47-9d98-1d803e128f52" containerName="test-curl-8066-087ad4"
Sep 12 02:12:31.524572 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.524572    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="395fdd38-01b8-4a47-9d98-1d803e128f52" containerName="test-curl-8066-087ad4"
Sep 12 02:12:31.524740 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.524652    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a6bdc5c-073f-4328-8d5f-612be56daa64" containerName="istio-proxy"
Sep 12 02:12:31.524740 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.524663    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ed9705-6d07-457a-aa38-8c4e3d9a68dc" containerName="istio-proxy"
Sep 12 02:12:31.524740 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.524671    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="395fdd38-01b8-4a47-9d98-1d803e128f52" containerName="test-curl-8066-087ad4"
Sep 12 02:12:31.527358 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.527334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.529903 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.529878    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a744d5\"/\"sub-test-model-a744d5-kserve-self-signed-certs\""
Sep 12 02:12:31.536570 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.536545    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"]
Sep 12 02:12:31.679072 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.679035    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-model-cache\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.679269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.679090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-home\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.679269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.679120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-tmp-dir\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.679269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.679156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ctll\" (UniqueName: \"kubernetes.io/projected/17c57349-d3ac-4b48-a124-a6c69426259c-kube-api-access-4ctll\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.679269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.679232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17c57349-d3ac-4b48-a124-a6c69426259c-tls-certs\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.679483 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.679282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-dshm\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.780076 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.779979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ctll\" (UniqueName: \"kubernetes.io/projected/17c57349-d3ac-4b48-a124-a6c69426259c-kube-api-access-4ctll\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.780076 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.780044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17c57349-d3ac-4b48-a124-a6c69426259c-tls-certs\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.780076 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.780068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-dshm\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.780540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.780259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-model-cache\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.780540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.780355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-home\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.780540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.780407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-tmp-dir\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.780712 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.780645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-model-cache\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.780712 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.780693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-home\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.780824 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.780717    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-tmp-dir\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.782560 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.782540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-dshm\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.782861 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.782843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17c57349-d3ac-4b48-a124-a6c69426259c-tls-certs\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.788224 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.788191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ctll\" (UniqueName: \"kubernetes.io/projected/17c57349-d3ac-4b48-a124-a6c69426259c-kube-api-access-4ctll\") pod \"sub-test-model-a744d5-kserve-cff76f99c-n8ncv\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:31.838419 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:31.838380    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:32.114524 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:32.114496    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"]
Sep 12 02:12:32.116331 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:12:32.116302    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17c57349_d3ac_4b48_a124_a6c69426259c.slice/crio-29ce7ab26081db9ccd2e885b16c7e56db89dc4064e8af45b1af79de17ae206e0 WatchSource:0}: Error finding container 29ce7ab26081db9ccd2e885b16c7e56db89dc4064e8af45b1af79de17ae206e0: Status 404 returned error can't find the container with id 29ce7ab26081db9ccd2e885b16c7e56db89dc4064e8af45b1af79de17ae206e0
Sep 12 02:12:32.485176 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:32.485082    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv" event={"ID":"17c57349-d3ac-4b48-a124-a6c69426259c","Type":"ContainerStarted","Data":"c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34"}
Sep 12 02:12:32.485176 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:32.485122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv" event={"ID":"17c57349-d3ac-4b48-a124-a6c69426259c","Type":"ContainerStarted","Data":"29ce7ab26081db9ccd2e885b16c7e56db89dc4064e8af45b1af79de17ae206e0"}
Sep 12 02:12:32.502934 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:32.502883    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv" podStartSLOduration=1.502868627 podStartE2EDuration="1.502868627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:12:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:12:32.501197471 +0000 UTC m=+1658.028098558" watchObservedRunningTime="2026-09-12 02:12:32.502868627 +0000 UTC m=+1658.029769734"
Sep 12 02:12:39.282317 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.282275    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp" podUID="c0e05f23-6882-409e-9db3-bf61ceac8a03" containerName="main" containerID="cri-o://c7b53dbfc6b0313b7e220fdcc22edeb2eca306a39275f0f9a3396ae8868e8ba0" gracePeriod=45
Sep 12 02:12:39.530507 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.530456    2580 generic.go:356] "Generic (PLEG): container finished" podID="c0e05f23-6882-409e-9db3-bf61ceac8a03" containerID="c7b53dbfc6b0313b7e220fdcc22edeb2eca306a39275f0f9a3396ae8868e8ba0" exitCode=0
Sep 12 02:12:39.530646 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.530521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp" event={"ID":"c0e05f23-6882-409e-9db3-bf61ceac8a03","Type":"ContainerDied","Data":"c7b53dbfc6b0313b7e220fdcc22edeb2eca306a39275f0f9a3396ae8868e8ba0"}
Sep 12 02:12:39.530646 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.530561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp" event={"ID":"c0e05f23-6882-409e-9db3-bf61ceac8a03","Type":"ContainerDied","Data":"76329b5b5080a44eee736a78fb5434397f1e7906022c1beb71642984fc7508ab"}
Sep 12 02:12:39.530646 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.530575    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76329b5b5080a44eee736a78fb5434397f1e7906022c1beb71642984fc7508ab"
Sep 12 02:12:39.539150 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.539097    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:12:39.654003 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.653971    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-model-cache\") pod \"c0e05f23-6882-409e-9db3-bf61ceac8a03\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") "
Sep 12 02:12:39.654186 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654071    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-tmp-dir\") pod \"c0e05f23-6882-409e-9db3-bf61ceac8a03\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") "
Sep 12 02:12:39.654186 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654115    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-dshm\") pod \"c0e05f23-6882-409e-9db3-bf61ceac8a03\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") "
Sep 12 02:12:39.654307 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654255    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-home\") pod \"c0e05f23-6882-409e-9db3-bf61ceac8a03\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") "
Sep 12 02:12:39.654348 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654321    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m76fj\" (UniqueName: \"kubernetes.io/projected/c0e05f23-6882-409e-9db3-bf61ceac8a03-kube-api-access-m76fj\") pod \"c0e05f23-6882-409e-9db3-bf61ceac8a03\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") "
Sep 12 02:12:39.654384 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654360    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0e05f23-6882-409e-9db3-bf61ceac8a03-tls-certs\") pod \"c0e05f23-6882-409e-9db3-bf61ceac8a03\" (UID: \"c0e05f23-6882-409e-9db3-bf61ceac8a03\") "
Sep 12 02:12:39.654384 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654347    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-model-cache" (OuterVolumeSpecName: "model-cache") pod "c0e05f23-6882-409e-9db3-bf61ceac8a03" (UID: "c0e05f23-6882-409e-9db3-bf61ceac8a03"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:12:39.654464 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654425    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c0e05f23-6882-409e-9db3-bf61ceac8a03" (UID: "c0e05f23-6882-409e-9db3-bf61ceac8a03"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:12:39.654543 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654517    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-home" (OuterVolumeSpecName: "home") pod "c0e05f23-6882-409e-9db3-bf61ceac8a03" (UID: "c0e05f23-6882-409e-9db3-bf61ceac8a03"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:12:39.654708 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654689    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:12:39.654765 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654708    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:12:39.654765 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.654718    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:12:39.656327 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.656305    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-dshm" (OuterVolumeSpecName: "dshm") pod "c0e05f23-6882-409e-9db3-bf61ceac8a03" (UID: "c0e05f23-6882-409e-9db3-bf61ceac8a03"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:12:39.656481 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.656446    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e05f23-6882-409e-9db3-bf61ceac8a03-kube-api-access-m76fj" (OuterVolumeSpecName: "kube-api-access-m76fj") pod "c0e05f23-6882-409e-9db3-bf61ceac8a03" (UID: "c0e05f23-6882-409e-9db3-bf61ceac8a03"). InnerVolumeSpecName "kube-api-access-m76fj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:12:39.656585 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.656505    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0e05f23-6882-409e-9db3-bf61ceac8a03-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c0e05f23-6882-409e-9db3-bf61ceac8a03" (UID: "c0e05f23-6882-409e-9db3-bf61ceac8a03"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:12:39.756168 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.756136    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m76fj\" (UniqueName: \"kubernetes.io/projected/c0e05f23-6882-409e-9db3-bf61ceac8a03-kube-api-access-m76fj\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:12:39.756168 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.756163    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0e05f23-6882-409e-9db3-bf61ceac8a03-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:12:39.756168 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:39.756172    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0e05f23-6882-409e-9db3-bf61ceac8a03-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:12:40.537718 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:40.537687    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"
Sep 12 02:12:40.559925 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:40.559886    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"]
Sep 12 02:12:40.564194 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:40.564166    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/auth-test-model-a744d5-kserve-66bb4cff87-hlrkp"]
Sep 12 02:12:40.983447 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:40.981241    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0e05f23-6882-409e-9db3-bf61ceac8a03" path="/var/lib/kubelet/pods/c0e05f23-6882-409e-9db3-bf61ceac8a03/volumes"
Sep 12 02:12:41.838910 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:41.838873    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:41.839338 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:41.838959    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:41.851134 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:41.851104    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:12:42.562267 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:12:42.562241    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:13:39.300356 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.300321    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8066-23884b"]
Sep 12 02:13:39.300765 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.300694    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0e05f23-6882-409e-9db3-bf61ceac8a03" containerName="main"
Sep 12 02:13:39.300765 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.300704    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0e05f23-6882-409e-9db3-bf61ceac8a03" containerName="main"
Sep 12 02:13:39.300843 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.300783    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0e05f23-6882-409e-9db3-bf61ceac8a03" containerName="main"
Sep 12 02:13:39.303467 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.303451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:39.305925 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.305904    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dng96\""
Sep 12 02:13:39.309403 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.309382    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8066-23884b"]
Sep 12 02:13:39.376346 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.376320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmqqs\" (UniqueName: \"kubernetes.io/projected/f4087a3f-5a79-4eba-8f63-489da93d257a-kube-api-access-tmqqs\") pod \"test-curl-8066-23884b\" (UID: \"f4087a3f-5a79-4eba-8f63-489da93d257a\") " pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:39.477245 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.477199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmqqs\" (UniqueName: \"kubernetes.io/projected/f4087a3f-5a79-4eba-8f63-489da93d257a-kube-api-access-tmqqs\") pod \"test-curl-8066-23884b\" (UID: \"f4087a3f-5a79-4eba-8f63-489da93d257a\") " pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:39.485563 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.485534    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmqqs\" (UniqueName: \"kubernetes.io/projected/f4087a3f-5a79-4eba-8f63-489da93d257a-kube-api-access-tmqqs\") pod \"test-curl-8066-23884b\" (UID: \"f4087a3f-5a79-4eba-8f63-489da93d257a\") " pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:39.614609 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.614535    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:39.884825 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.884800    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8066-23884b"]
Sep 12 02:13:39.886988 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:13:39.886956    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4087a3f_5a79_4eba_8f63_489da93d257a.slice/crio-174e4a22f919fabb35437a239749d46e183ac05d3e3ce0d0afeb30694dea7ee8 WatchSource:0}: Error finding container 174e4a22f919fabb35437a239749d46e183ac05d3e3ce0d0afeb30694dea7ee8: Status 404 returned error can't find the container with id 174e4a22f919fabb35437a239749d46e183ac05d3e3ce0d0afeb30694dea7ee8
Sep 12 02:13:39.898063 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:39.898039    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8066-23884b" event={"ID":"f4087a3f-5a79-4eba-8f63-489da93d257a","Type":"ContainerStarted","Data":"174e4a22f919fabb35437a239749d46e183ac05d3e3ce0d0afeb30694dea7ee8"}
Sep 12 02:13:40.906983 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:40.906949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8066-23884b" event={"ID":"f4087a3f-5a79-4eba-8f63-489da93d257a","Type":"ContainerStarted","Data":"f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0"}
Sep 12 02:13:40.922732 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:40.922679    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8066-23884b" podStartSLOduration=1.922665828 podStartE2EDuration="1.922665828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:13:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:13:40.92101918 +0000 UTC m=+1726.447920258" watchObservedRunningTime="2026-09-12 02:13:40.922665828 +0000 UTC m=+1726.449566906"
Sep 12 02:13:41.503132 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.503097    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8066-23884b"]
Sep 12 02:13:41.505946 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.505919    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8066-23884b"]
Sep 12 02:13:41.628268 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.628178    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8066-b13c03"]
Sep 12 02:13:41.628564 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.628552    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" containerName="test-curl-8066-23884b"
Sep 12 02:13:41.628602 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.628566    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" containerName="test-curl-8066-23884b"
Sep 12 02:13:41.628637 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.628632    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" containerName="test-curl-8066-23884b"
Sep 12 02:13:41.631600 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.631579    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:41.637630 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.637605    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8066-b13c03"]
Sep 12 02:13:41.698976 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.698948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg7k2\" (UniqueName: \"kubernetes.io/projected/3e0efdea-0b40-4ee3-a45f-d80091a87b50-kube-api-access-bg7k2\") pod \"test-curl-8066-b13c03\" (UID: \"3e0efdea-0b40-4ee3-a45f-d80091a87b50\") " pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:41.800092 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.800066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg7k2\" (UniqueName: \"kubernetes.io/projected/3e0efdea-0b40-4ee3-a45f-d80091a87b50-kube-api-access-bg7k2\") pod \"test-curl-8066-b13c03\" (UID: \"3e0efdea-0b40-4ee3-a45f-d80091a87b50\") " pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:41.809852 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.809829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg7k2\" (UniqueName: \"kubernetes.io/projected/3e0efdea-0b40-4ee3-a45f-d80091a87b50-kube-api-access-bg7k2\") pod \"test-curl-8066-b13c03\" (UID: \"3e0efdea-0b40-4ee3-a45f-d80091a87b50\") " pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:41.912766 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:41.912694    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8066-23884b" container="test-curl-8066-23884b"
Sep 12 02:13:41.942316 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:41.942295    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:42.210896 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:42.210870    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8066-b13c03"]
Sep 12 02:13:42.921312 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:42.921275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8066-b13c03" event={"ID":"3e0efdea-0b40-4ee3-a45f-d80091a87b50","Type":"ContainerStarted","Data":"ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7"}
Sep 12 02:13:42.921776 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:42.921324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8066-b13c03" event={"ID":"3e0efdea-0b40-4ee3-a45f-d80091a87b50","Type":"ContainerStarted","Data":"bbaf2ba5de331bd7ca7c5ee7bca6df8eee5d7ed9876877e808931b1aa84cc27d"}
Sep 12 02:13:42.921776 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:42.921480    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8066-23884b" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" containerName="test-curl-8066-23884b" containerID="cri-o://f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0" gracePeriod=2
Sep 12 02:13:42.936196 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:42.936161    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8066-23884b\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:42.936196 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:42.936169    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8066-b13c03" podStartSLOduration=1.936154666 podStartE2EDuration="1.936154666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:13:42.934247153 +0000 UTC m=+1728.461148231" watchObservedRunningTime="2026-09-12 02:13:42.936154666 +0000 UTC m=+1728.463055745"
Sep 12 02:13:43.521702 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:43.521671    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8066-b13c03"]
Sep 12 02:13:43.525178 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:43.525152    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8066-b13c03"]
Sep 12 02:13:43.927301 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:43.927269    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8066-b13c03" container="test-curl-8066-b13c03"
Sep 12 02:13:44.932930 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:44.932896    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8066-b13c03" podUID="3e0efdea-0b40-4ee3-a45f-d80091a87b50" containerName="test-curl-8066-b13c03" containerID="cri-o://ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7" gracePeriod=2
Sep 12 02:13:44.935341 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:44.935307    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8066-b13c03\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="3e0efdea-0b40-4ee3-a45f-d80091a87b50" pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:44.978336 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:44.978297    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8066-b13c03\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="3e0efdea-0b40-4ee3-a45f-d80091a87b50" pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:45.069726 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.069701    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:45.071847 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:45.071818    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8066-23884b\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:45.129356 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.129334    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmqqs\" (UniqueName: \"kubernetes.io/projected/f4087a3f-5a79-4eba-8f63-489da93d257a-kube-api-access-tmqqs\") pod \"f4087a3f-5a79-4eba-8f63-489da93d257a\" (UID: \"f4087a3f-5a79-4eba-8f63-489da93d257a\") "
Sep 12 02:13:45.131249 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.131222    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4087a3f-5a79-4eba-8f63-489da93d257a-kube-api-access-tmqqs" (OuterVolumeSpecName: "kube-api-access-tmqqs") pod "f4087a3f-5a79-4eba-8f63-489da93d257a" (UID: "f4087a3f-5a79-4eba-8f63-489da93d257a"). InnerVolumeSpecName "kube-api-access-tmqqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:13:45.230549 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.230455    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmqqs\" (UniqueName: \"kubernetes.io/projected/f4087a3f-5a79-4eba-8f63-489da93d257a-kube-api-access-tmqqs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:13:45.507973 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.507939    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"]
Sep 12 02:13:45.526227 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:45.526181    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8066-23884b\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:45.939677 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.939579    2580 generic.go:356] "Generic (PLEG): container finished" podID="f4087a3f-5a79-4eba-8f63-489da93d257a" containerID="f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0" exitCode=137
Sep 12 02:13:45.939677 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.939623    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:45.940125 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.939685    2580 scope.go:117] "RemoveContainer" containerID="f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0"
Sep 12 02:13:45.942073 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:45.942040    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8066-23884b\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:45.948885 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.948869    2580 scope.go:117] "RemoveContainer" containerID="f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0"
Sep 12 02:13:45.949116 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:45.949099    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0\": container with ID starting with f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0 not found: ID does not exist" containerID="f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0"
Sep 12 02:13:45.949173 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:45.949124    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0"} err="failed to get container status \"f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0\": rpc error: code = NotFound desc = could not find container \"f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0\": container with ID starting with f892fc0fb7c0ca8f896612064899524d09e674528c59daba3d32dfd860932bd0 not found: ID does not exist"
Sep 12 02:13:45.949506 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:45.949484    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8066-23884b\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" pod="opendatahub/test-curl-8066-23884b"
Sep 12 02:13:46.978913 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:46.978877    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4087a3f-5a79-4eba-8f63-489da93d257a" path="/var/lib/kubelet/pods/f4087a3f-5a79-4eba-8f63-489da93d257a/volumes"
Sep 12 02:13:47.134674 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:47.134648    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:47.253709 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:47.253680    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bg7k2\" (UniqueName: \"kubernetes.io/projected/3e0efdea-0b40-4ee3-a45f-d80091a87b50-kube-api-access-bg7k2\") pod \"3e0efdea-0b40-4ee3-a45f-d80091a87b50\" (UID: \"3e0efdea-0b40-4ee3-a45f-d80091a87b50\") "
Sep 12 02:13:47.255822 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:47.255798    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e0efdea-0b40-4ee3-a45f-d80091a87b50-kube-api-access-bg7k2" (OuterVolumeSpecName: "kube-api-access-bg7k2") pod "3e0efdea-0b40-4ee3-a45f-d80091a87b50" (UID: "3e0efdea-0b40-4ee3-a45f-d80091a87b50"). InnerVolumeSpecName "kube-api-access-bg7k2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:13:47.355272 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:47.355241    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg7k2\" (UniqueName: \"kubernetes.io/projected/3e0efdea-0b40-4ee3-a45f-d80091a87b50-kube-api-access-bg7k2\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:13:47.962005 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:47.961971    2580 generic.go:356] "Generic (PLEG): container finished" podID="3e0efdea-0b40-4ee3-a45f-d80091a87b50" containerID="ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7" exitCode=137
Sep 12 02:13:47.962186 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:47.962015    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:47.962186 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:47.962061    2580 scope.go:117] "RemoveContainer" containerID="ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7"
Sep 12 02:13:47.971639 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:47.971619    2580 scope.go:117] "RemoveContainer" containerID="ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7"
Sep 12 02:13:47.971931 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:47.971911    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7\": container with ID starting with ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7 not found: ID does not exist" containerID="ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7"
Sep 12 02:13:47.971998 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:47.971939    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7"} err="failed to get container status \"ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7\": rpc error: code = NotFound desc = could not find container \"ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7\": container with ID starting with ed79dcf41da9baa35fe55aa036ef4dc60c099005f7771d27d553eba070e6fbd7 not found: ID does not exist"
Sep 12 02:13:47.972417 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:13:47.972393    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8066-b13c03\" is forbidden: User \"system:node:ip-10-0-143-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-251.ec2.internal' and this object" podUID="3e0efdea-0b40-4ee3-a45f-d80091a87b50" pod="opendatahub/test-curl-8066-b13c03"
Sep 12 02:13:48.979119 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:48.979084    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e0efdea-0b40-4ee3-a45f-d80091a87b50" path="/var/lib/kubelet/pods/3e0efdea-0b40-4ee3-a45f-d80091a87b50/volumes"
Sep 12 02:13:55.611884 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:55.611854    2580 scope.go:117] "RemoveContainer" containerID="c7b53dbfc6b0313b7e220fdcc22edeb2eca306a39275f0f9a3396ae8868e8ba0"
Sep 12 02:13:55.620414 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:13:55.620391    2580 scope.go:117] "RemoveContainer" containerID="5b0c39561c09f28f239f3a04c601b4a3c4d5769171758022e7f810190a352eed"
Sep 12 02:14:00.529840 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.529800    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv" podUID="17c57349-d3ac-4b48-a124-a6c69426259c" containerName="main" containerID="cri-o://c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34" gracePeriod=45
Sep 12 02:14:00.790826 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.790760    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:14:00.879611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.879576    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-model-cache\") pod \"17c57349-d3ac-4b48-a124-a6c69426259c\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") "
Sep 12 02:14:00.879611 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.879616    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17c57349-d3ac-4b48-a124-a6c69426259c-tls-certs\") pod \"17c57349-d3ac-4b48-a124-a6c69426259c\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") "
Sep 12 02:14:00.879860 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.879658    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ctll\" (UniqueName: \"kubernetes.io/projected/17c57349-d3ac-4b48-a124-a6c69426259c-kube-api-access-4ctll\") pod \"17c57349-d3ac-4b48-a124-a6c69426259c\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") "
Sep 12 02:14:00.879860 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.879676    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-dshm\") pod \"17c57349-d3ac-4b48-a124-a6c69426259c\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") "
Sep 12 02:14:00.879860 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.879695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-home\") pod \"17c57349-d3ac-4b48-a124-a6c69426259c\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") "
Sep 12 02:14:00.879860 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.879738    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-tmp-dir\") pod \"17c57349-d3ac-4b48-a124-a6c69426259c\" (UID: \"17c57349-d3ac-4b48-a124-a6c69426259c\") "
Sep 12 02:14:00.880059 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.879859    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-model-cache" (OuterVolumeSpecName: "model-cache") pod "17c57349-d3ac-4b48-a124-a6c69426259c" (UID: "17c57349-d3ac-4b48-a124-a6c69426259c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:14:00.880059 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.879960    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-home" (OuterVolumeSpecName: "home") pod "17c57349-d3ac-4b48-a124-a6c69426259c" (UID: "17c57349-d3ac-4b48-a124-a6c69426259c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:14:00.880059 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.880006    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:14:00.880059 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.880034    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "17c57349-d3ac-4b48-a124-a6c69426259c" (UID: "17c57349-d3ac-4b48-a124-a6c69426259c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:14:00.881816 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.881791    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-dshm" (OuterVolumeSpecName: "dshm") pod "17c57349-d3ac-4b48-a124-a6c69426259c" (UID: "17c57349-d3ac-4b48-a124-a6c69426259c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:14:00.881816 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.881812    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17c57349-d3ac-4b48-a124-a6c69426259c-kube-api-access-4ctll" (OuterVolumeSpecName: "kube-api-access-4ctll") pod "17c57349-d3ac-4b48-a124-a6c69426259c" (UID: "17c57349-d3ac-4b48-a124-a6c69426259c"). InnerVolumeSpecName "kube-api-access-4ctll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:14:00.881986 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.881830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17c57349-d3ac-4b48-a124-a6c69426259c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "17c57349-d3ac-4b48-a124-a6c69426259c" (UID: "17c57349-d3ac-4b48-a124-a6c69426259c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:14:00.980980 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.980949    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ctll\" (UniqueName: \"kubernetes.io/projected/17c57349-d3ac-4b48-a124-a6c69426259c-kube-api-access-4ctll\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:14:00.980980 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.980975    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:14:00.980980 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.980985    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:14:00.981198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.980993    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c57349-d3ac-4b48-a124-a6c69426259c-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:14:00.981198 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:00.981001    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17c57349-d3ac-4b48-a124-a6c69426259c-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:14:01.044593 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:01.044505    2580 generic.go:356] "Generic (PLEG): container finished" podID="17c57349-d3ac-4b48-a124-a6c69426259c" containerID="c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34" exitCode=0
Sep 12 02:14:01.044593 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:01.044553    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"
Sep 12 02:14:01.044812 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:01.044586    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv" event={"ID":"17c57349-d3ac-4b48-a124-a6c69426259c","Type":"ContainerDied","Data":"c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34"}
Sep 12 02:14:01.044812 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:01.044633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv" event={"ID":"17c57349-d3ac-4b48-a124-a6c69426259c","Type":"ContainerDied","Data":"29ce7ab26081db9ccd2e885b16c7e56db89dc4064e8af45b1af79de17ae206e0"}
Sep 12 02:14:01.044812 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:01.044652    2580 scope.go:117] "RemoveContainer" containerID="c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34"
Sep 12 02:14:01.054033 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:01.054014    2580 scope.go:117] "RemoveContainer" containerID="c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34"
Sep 12 02:14:01.054282 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:14:01.054265    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34\": container with ID starting with c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34 not found: ID does not exist" containerID="c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34"
Sep 12 02:14:01.054356 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:01.054290    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34"} err="failed to get container status \"c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34\": rpc error: code = NotFound desc = could not find container \"c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34\": container with ID starting with c2eff2e27b637e5dff2fd5525d378203f07cbc325939235d006d3fa90c372f34 not found: ID does not exist"
Sep 12 02:14:01.060370 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:01.060348    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"]
Sep 12 02:14:01.063620 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:01.063600    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/sub-test-model-a744d5-kserve-cff76f99c-n8ncv"]
Sep 12 02:14:02.980929 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:02.980894    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17c57349-d3ac-4b48-a124-a6c69426259c" path="/var/lib/kubelet/pods/17c57349-d3ac-4b48-a124-a6c69426259c/volumes"
Sep 12 02:14:03.529733 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.529696    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"]
Sep 12 02:14:03.530098 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.530082    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e0efdea-0b40-4ee3-a45f-d80091a87b50" containerName="test-curl-8066-b13c03"
Sep 12 02:14:03.530149 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.530103    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e0efdea-0b40-4ee3-a45f-d80091a87b50" containerName="test-curl-8066-b13c03"
Sep 12 02:14:03.530149 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.530140    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17c57349-d3ac-4b48-a124-a6c69426259c" containerName="main"
Sep 12 02:14:03.530228 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.530150    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="17c57349-d3ac-4b48-a124-a6c69426259c" containerName="main"
Sep 12 02:14:03.530285 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.530273    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="17c57349-d3ac-4b48-a124-a6c69426259c" containerName="main"
Sep 12 02:14:03.530324 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.530291    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e0efdea-0b40-4ee3-a45f-d80091a87b50" containerName="test-curl-8066-b13c03"
Sep 12 02:14:03.534568 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.534543    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.538533 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.538508    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-a744d5\"/\"kube-root-ca.crt\""
Sep 12 02:14:03.538667 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.538538    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a744d5\"/\"rate-test-model-a744d5-kserve-self-signed-certs\""
Sep 12 02:14:03.538667 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.538554    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a744d5\"/\"default-dockercfg-fb9kj\""
Sep 12 02:14:03.538667 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.538538    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-a744d5\"/\"openshift-service-ca.crt\""
Sep 12 02:14:03.541530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.541486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"]
Sep 12 02:14:03.704857 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.704821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-dshm\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.705067 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.704869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-model-cache\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.705067 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.704964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tmp-dir\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.705067 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.705014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-home\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.705067 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.705046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsn9z\" (UniqueName: \"kubernetes.io/projected/43563f46-c3d8-4bbf-a7e7-a836b4a07016-kube-api-access-xsn9z\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.705317 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.705081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tls-certs\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.805724 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.805624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-home\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.805724 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.805685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsn9z\" (UniqueName: \"kubernetes.io/projected/43563f46-c3d8-4bbf-a7e7-a836b4a07016-kube-api-access-xsn9z\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.805959 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.805730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tls-certs\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.805959 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.805780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-dshm\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.805959 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.805810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-model-cache\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.805959 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.805883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tmp-dir\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.806182 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.806052    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-home\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.806286 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.806232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-model-cache\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.806348 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.806305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tmp-dir\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.808074 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.808045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-dshm\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.808393 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.808372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tls-certs\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.813670 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.813648    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsn9z\" (UniqueName: \"kubernetes.io/projected/43563f46-c3d8-4bbf-a7e7-a836b4a07016-kube-api-access-xsn9z\") pod \"rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") " pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:03.846438 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:03.846405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:04.119462 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:04.119430    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"]
Sep 12 02:14:04.120196 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:14:04.120172    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43563f46_c3d8_4bbf_a7e7_a836b4a07016.slice/crio-0e2c574455413ebde4c7ff39af2a5bf114e465ed39a60874f51f606f22f866ec WatchSource:0}: Error finding container 0e2c574455413ebde4c7ff39af2a5bf114e465ed39a60874f51f606f22f866ec: Status 404 returned error can't find the container with id 0e2c574455413ebde4c7ff39af2a5bf114e465ed39a60874f51f606f22f866ec
Sep 12 02:14:05.071502 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:05.071460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7" event={"ID":"43563f46-c3d8-4bbf-a7e7-a836b4a07016","Type":"ContainerStarted","Data":"439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d"}
Sep 12 02:14:05.071684 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:05.071509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7" event={"ID":"43563f46-c3d8-4bbf-a7e7-a836b4a07016","Type":"ContainerStarted","Data":"0e2c574455413ebde4c7ff39af2a5bf114e465ed39a60874f51f606f22f866ec"}
Sep 12 02:14:05.093846 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:05.093801    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7" podStartSLOduration=2.093787222 podStartE2EDuration="2.093787222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:14:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:14:05.090505987 +0000 UTC m=+1750.617407067" watchObservedRunningTime="2026-09-12 02:14:05.093787222 +0000 UTC m=+1750.620688301"
Sep 12 02:14:13.847411 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:13.847366    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:13.847774 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:13.847435    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:13.859965 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:13.859943    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:14.133038 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:14.132946    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:14:55.164312 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:55.164285    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:14:55.178565 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:14:55.178543    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:15:00.132289 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.132255    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn"]
Sep 12 02:15:00.135888 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.135863    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr"]
Sep 12 02:15:00.136028 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.135968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn"
Sep 12 02:15:00.138394 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.138375    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:15:00.139084 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.139069    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr"
Sep 12 02:15:00.145727 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.145702    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn"]
Sep 12 02:15:00.147946 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.147927    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn"
Sep 12 02:15:00.152070 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.151278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr"]
Sep 12 02:15:00.157157 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.157140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr"
Sep 12 02:15:00.429452 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.429403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn"]
Sep 12 02:15:00.430878 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:15:00.430851    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f3e447_6326_41c3_b0ab_a30a37ec53a7.slice/crio-bbec2585ccb9c2478d37c9901b2a3f7dae12281c2f45aa6a1c14d2fd5905a484 WatchSource:0}: Error finding container bbec2585ccb9c2478d37c9901b2a3f7dae12281c2f45aa6a1c14d2fd5905a484: Status 404 returned error can't find the container with id bbec2585ccb9c2478d37c9901b2a3f7dae12281c2f45aa6a1c14d2fd5905a484
Sep 12 02:15:00.440290 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:00.440266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr"]
Sep 12 02:15:00.444416 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:15:00.444393    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb274df06_5749_4259_a56d_2ba1e92aaf1f.slice/crio-e984d8bf5762d72f168035688674cddc600e0803b43a53445ac51da23c0bb896 WatchSource:0}: Error finding container e984d8bf5762d72f168035688674cddc600e0803b43a53445ac51da23c0bb896: Status 404 returned error can't find the container with id e984d8bf5762d72f168035688674cddc600e0803b43a53445ac51da23c0bb896
Sep 12 02:15:01.397587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:01.397547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr" event={"ID":"b274df06-5749-4259-a56d-2ba1e92aaf1f","Type":"ContainerStarted","Data":"fe8da88ddbcd180b51adf2de448be830f4e1d2d9cbb2fa6443452d91bce78738"}
Sep 12 02:15:01.398012 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:01.397612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr" event={"ID":"b274df06-5749-4259-a56d-2ba1e92aaf1f","Type":"ContainerStarted","Data":"e984d8bf5762d72f168035688674cddc600e0803b43a53445ac51da23c0bb896"}
Sep 12 02:15:01.399138 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:01.399107    2580 generic.go:356] "Generic (PLEG): container finished" podID="62f3e447-6326-41c3-b0ab-a30a37ec53a7" containerID="2e92bc0526b725fb07671461661791cab16430466c8b0a2591d42dab935fa7cb" exitCode=0
Sep 12 02:15:01.399296 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:01.399172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn" event={"ID":"62f3e447-6326-41c3-b0ab-a30a37ec53a7","Type":"ContainerDied","Data":"2e92bc0526b725fb07671461661791cab16430466c8b0a2591d42dab935fa7cb"}
Sep 12 02:15:01.399296 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:01.399248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn" event={"ID":"62f3e447-6326-41c3-b0ab-a30a37ec53a7","Type":"ContainerStarted","Data":"bbec2585ccb9c2478d37c9901b2a3f7dae12281c2f45aa6a1c14d2fd5905a484"}
Sep 12 02:15:01.412185 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:01.412144    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr" podStartSLOduration=1.412132108 podStartE2EDuration="1.412132108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:15:01.411316232 +0000 UTC m=+1806.938217325" watchObservedRunningTime="2026-09-12 02:15:01.412132108 +0000 UTC m=+1806.939033186"
Sep 12 02:15:02.406289 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:02.406254    2580 generic.go:356] "Generic (PLEG): container finished" podID="b274df06-5749-4259-a56d-2ba1e92aaf1f" containerID="fe8da88ddbcd180b51adf2de448be830f4e1d2d9cbb2fa6443452d91bce78738" exitCode=0
Sep 12 02:15:02.406691 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:02.406351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr" event={"ID":"b274df06-5749-4259-a56d-2ba1e92aaf1f","Type":"ContainerDied","Data":"fe8da88ddbcd180b51adf2de448be830f4e1d2d9cbb2fa6443452d91bce78738"}
Sep 12 02:15:02.542039 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:02.542015    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn"
Sep 12 02:15:03.413572 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:03.413535    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn" event={"ID":"62f3e447-6326-41c3-b0ab-a30a37ec53a7","Type":"ContainerDied","Data":"bbec2585ccb9c2478d37c9901b2a3f7dae12281c2f45aa6a1c14d2fd5905a484"}
Sep 12 02:15:03.413895 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:03.413596    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbec2585ccb9c2478d37c9901b2a3f7dae12281c2f45aa6a1c14d2fd5905a484"
Sep 12 02:15:03.413895 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:03.413554    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn"
Sep 12 02:15:03.544678 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:03.544657    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr"
Sep 12 02:15:04.421108 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:04.421080    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr"
Sep 12 02:15:04.421505 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:04.421080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819655-rszcr" event={"ID":"b274df06-5749-4259-a56d-2ba1e92aaf1f","Type":"ContainerDied","Data":"e984d8bf5762d72f168035688674cddc600e0803b43a53445ac51da23c0bb896"}
Sep 12 02:15:04.421505 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:04.421201    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e984d8bf5762d72f168035688674cddc600e0803b43a53445ac51da23c0bb896"
Sep 12 02:15:30.304492 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:30.304458    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"]
Sep 12 02:15:37.182081 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.182040    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"]
Sep 12 02:15:37.182552 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.182406    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f3e447-6326-41c3-b0ab-a30a37ec53a7" containerName="cleanup"
Sep 12 02:15:37.182552 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.182417    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f3e447-6326-41c3-b0ab-a30a37ec53a7" containerName="cleanup"
Sep 12 02:15:37.182552 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.182428    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b274df06-5749-4259-a56d-2ba1e92aaf1f" containerName="cleanup"
Sep 12 02:15:37.182552 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.182433    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b274df06-5749-4259-a56d-2ba1e92aaf1f" containerName="cleanup"
Sep 12 02:15:37.182552 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.182504    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b274df06-5749-4259-a56d-2ba1e92aaf1f" containerName="cleanup"
Sep 12 02:15:37.182552 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.182514    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f3e447-6326-41c3-b0ab-a30a37ec53a7" containerName="cleanup"
Sep 12 02:15:37.185748 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.185719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:37.188235 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.188183    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-103dc5\""
Sep 12 02:15:37.188356 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.188289    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-103dc5-dockercfg-7hw54\""
Sep 12 02:15:37.194477 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.194449    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"]
Sep 12 02:15:37.306540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.306503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9gqq\" (UniqueName: \"kubernetes.io/projected/e52a2915-0f73-490f-a1c1-001ecd61da6f-kube-api-access-t9gqq\") pod \"payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k\" (UID: \"e52a2915-0f73-490f-a1c1-001ecd61da6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:37.306540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.306551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52a2915-0f73-490f-a1c1-001ecd61da6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k\" (UID: \"e52a2915-0f73-490f-a1c1-001ecd61da6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:37.333055 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.333024    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"]
Sep 12 02:15:37.336456 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.336433    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:15:37.353958 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.353935    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"]
Sep 12 02:15:37.407736 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.407706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9gqq\" (UniqueName: \"kubernetes.io/projected/e52a2915-0f73-490f-a1c1-001ecd61da6f-kube-api-access-t9gqq\") pod \"payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k\" (UID: \"e52a2915-0f73-490f-a1c1-001ecd61da6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:37.407886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.407743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52a2915-0f73-490f-a1c1-001ecd61da6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k\" (UID: \"e52a2915-0f73-490f-a1c1-001ecd61da6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:37.407886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.407774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/036b809b-3e67-492b-b985-4447bb54be9d-kube-api-access-q6x68\") pod \"payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj\" (UID: \"036b809b-3e67-492b-b985-4447bb54be9d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:15:37.407886 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.407802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/036b809b-3e67-492b-b985-4447bb54be9d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj\" (UID: \"036b809b-3e67-492b-b985-4447bb54be9d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:15:37.408443 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.408414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52a2915-0f73-490f-a1c1-001ecd61da6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k\" (UID: \"e52a2915-0f73-490f-a1c1-001ecd61da6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:37.419731 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.419711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9gqq\" (UniqueName: \"kubernetes.io/projected/e52a2915-0f73-490f-a1c1-001ecd61da6f-kube-api-access-t9gqq\") pod \"payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k\" (UID: \"e52a2915-0f73-490f-a1c1-001ecd61da6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:37.497476 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.497452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:37.508574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.508547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/036b809b-3e67-492b-b985-4447bb54be9d-kube-api-access-q6x68\") pod \"payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj\" (UID: \"036b809b-3e67-492b-b985-4447bb54be9d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:15:37.508674 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.508577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/036b809b-3e67-492b-b985-4447bb54be9d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj\" (UID: \"036b809b-3e67-492b-b985-4447bb54be9d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:15:37.509541 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.509521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/036b809b-3e67-492b-b985-4447bb54be9d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj\" (UID: \"036b809b-3e67-492b-b985-4447bb54be9d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:15:37.518830 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.518810    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/036b809b-3e67-492b-b985-4447bb54be9d-kube-api-access-q6x68\") pod \"payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj\" (UID: \"036b809b-3e67-492b-b985-4447bb54be9d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:15:37.645871 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.645834    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:15:37.768589 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.768528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"]
Sep 12 02:15:37.770082 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:15:37.770053    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode52a2915_0f73_490f_a1c1_001ecd61da6f.slice/crio-5de6a2b4de4e8ec2b29722705a9fd74bb7a9fcc14bc69b885d59270bd17bb70f WatchSource:0}: Error finding container 5de6a2b4de4e8ec2b29722705a9fd74bb7a9fcc14bc69b885d59270bd17bb70f: Status 404 returned error can't find the container with id 5de6a2b4de4e8ec2b29722705a9fd74bb7a9fcc14bc69b885d59270bd17bb70f
Sep 12 02:15:37.919711 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:37.919688    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"]
Sep 12 02:15:38.628590 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:38.628554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj" event={"ID":"036b809b-3e67-492b-b985-4447bb54be9d","Type":"ContainerStarted","Data":"f809fc25e850036f3ec8b983019bbb357cda72e99c49dd988bba9488a749fc19"}
Sep 12 02:15:38.629026 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:38.628609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj" event={"ID":"036b809b-3e67-492b-b985-4447bb54be9d","Type":"ContainerStarted","Data":"2a6c0e1aea5c0d785a9c94ef83635e07c719e3baf24d71a182ee33a7f0bd6f17"}
Sep 12 02:15:38.629026 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:38.628641    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:15:38.630264 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:38.630234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k" event={"ID":"e52a2915-0f73-490f-a1c1-001ecd61da6f","Type":"ContainerStarted","Data":"c6fe45d75da5be874ef335bf444983447dd2a1f5f77cfc8e37ccb19fb534b691"}
Sep 12 02:15:38.630390 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:38.630297    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:38.630390 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:38.630323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k" event={"ID":"e52a2915-0f73-490f-a1c1-001ecd61da6f","Type":"ContainerStarted","Data":"5de6a2b4de4e8ec2b29722705a9fd74bb7a9fcc14bc69b885d59270bd17bb70f"}
Sep 12 02:15:38.645459 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:38.645390    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj" podStartSLOduration=1.645377214 podStartE2EDuration="1.645377214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:15:38.642870663 +0000 UTC m=+1844.169771742" watchObservedRunningTime="2026-09-12 02:15:38.645377214 +0000 UTC m=+1844.172278292"
Sep 12 02:15:38.657832 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:38.657791    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k" podStartSLOduration=1.65777911 podStartE2EDuration="1.65777911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:15:38.656960931 +0000 UTC m=+1844.183862000" watchObservedRunningTime="2026-09-12 02:15:38.65777911 +0000 UTC m=+1844.184680188"
Sep 12 02:15:45.595555 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.595508    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7" podUID="43563f46-c3d8-4bbf-a7e7-a836b4a07016" containerName="main" containerID="cri-o://439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d" gracePeriod=45
Sep 12 02:15:45.850350 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.850292    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:15:45.984478 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984439    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tls-certs\") pod \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") "
Sep 12 02:15:45.984655 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984491    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsn9z\" (UniqueName: \"kubernetes.io/projected/43563f46-c3d8-4bbf-a7e7-a836b4a07016-kube-api-access-xsn9z\") pod \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") "
Sep 12 02:15:45.984655 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984562    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tmp-dir\") pod \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") "
Sep 12 02:15:45.984655 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984594    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-model-cache\") pod \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") "
Sep 12 02:15:45.984655 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984629    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-home\") pod \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") "
Sep 12 02:15:45.984875 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984668    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-dshm\") pod \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\" (UID: \"43563f46-c3d8-4bbf-a7e7-a836b4a07016\") "
Sep 12 02:15:45.984933 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984868    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43563f46-c3d8-4bbf-a7e7-a836b4a07016" (UID: "43563f46-c3d8-4bbf-a7e7-a836b4a07016"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:15:45.984933 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984875    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-model-cache" (OuterVolumeSpecName: "model-cache") pod "43563f46-c3d8-4bbf-a7e7-a836b4a07016" (UID: "43563f46-c3d8-4bbf-a7e7-a836b4a07016"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:15:45.985036 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984980    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-home" (OuterVolumeSpecName: "home") pod "43563f46-c3d8-4bbf-a7e7-a836b4a07016" (UID: "43563f46-c3d8-4bbf-a7e7-a836b4a07016"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:15:45.985036 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.984987    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:15:45.985036 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.985029    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:15:45.986661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.986629    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43563f46-c3d8-4bbf-a7e7-a836b4a07016" (UID: "43563f46-c3d8-4bbf-a7e7-a836b4a07016"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:15:45.986661 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.986653    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-dshm" (OuterVolumeSpecName: "dshm") pod "43563f46-c3d8-4bbf-a7e7-a836b4a07016" (UID: "43563f46-c3d8-4bbf-a7e7-a836b4a07016"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:15:45.986812 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:45.986689    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43563f46-c3d8-4bbf-a7e7-a836b4a07016-kube-api-access-xsn9z" (OuterVolumeSpecName: "kube-api-access-xsn9z") pod "43563f46-c3d8-4bbf-a7e7-a836b4a07016" (UID: "43563f46-c3d8-4bbf-a7e7-a836b4a07016"). InnerVolumeSpecName "kube-api-access-xsn9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:15:46.085956 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.085923    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43563f46-c3d8-4bbf-a7e7-a836b4a07016-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:15:46.085956 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.085957    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsn9z\" (UniqueName: \"kubernetes.io/projected/43563f46-c3d8-4bbf-a7e7-a836b4a07016-kube-api-access-xsn9z\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:15:46.086125 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.085972    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:15:46.086125 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.085985    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43563f46-c3d8-4bbf-a7e7-a836b4a07016-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:15:46.677568 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.677536    2580 generic.go:356] "Generic (PLEG): container finished" podID="43563f46-c3d8-4bbf-a7e7-a836b4a07016" containerID="439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d" exitCode=0
Sep 12 02:15:46.677985 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.677592    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"
Sep 12 02:15:46.677985 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.677629    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7" event={"ID":"43563f46-c3d8-4bbf-a7e7-a836b4a07016","Type":"ContainerDied","Data":"439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d"}
Sep 12 02:15:46.677985 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.677674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7" event={"ID":"43563f46-c3d8-4bbf-a7e7-a836b4a07016","Type":"ContainerDied","Data":"0e2c574455413ebde4c7ff39af2a5bf114e465ed39a60874f51f606f22f866ec"}
Sep 12 02:15:46.677985 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.677692    2580 scope.go:117] "RemoveContainer" containerID="439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d"
Sep 12 02:15:46.687600 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.687525    2580 scope.go:117] "RemoveContainer" containerID="439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d"
Sep 12 02:15:46.687839 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:15:46.687812    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d\": container with ID starting with 439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d not found: ID does not exist" containerID="439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d"
Sep 12 02:15:46.687931 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.687849    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d"} err="failed to get container status \"439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d\": rpc error: code = NotFound desc = could not find container \"439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d\": container with ID starting with 439f97df46fd7aaf145582664714f4d21a718db40755679ab9f7f9e58e68a69d not found: ID does not exist"
Sep 12 02:15:46.703259 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.703234    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"]
Sep 12 02:15:46.706388 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.706369    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a744d5/rate-test-model-a744d5-kserve-5bd4d76d55-qv7p7"]
Sep 12 02:15:46.981777 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:46.981698    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43563f46-c3d8-4bbf-a7e7-a836b4a07016" path="/var/lib/kubelet/pods/43563f46-c3d8-4bbf-a7e7-a836b4a07016/volumes"
Sep 12 02:15:49.638884 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:49.638851    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:15:49.639268 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:15:49.638918    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:16:06.971132 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:06.971090    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"]
Sep 12 02:16:06.973591 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:06.971729    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43563f46-c3d8-4bbf-a7e7-a836b4a07016" containerName="main"
Sep 12 02:16:06.973591 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:06.971747    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="43563f46-c3d8-4bbf-a7e7-a836b4a07016" containerName="main"
Sep 12 02:16:06.973591 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:06.971855    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="43563f46-c3d8-4bbf-a7e7-a836b4a07016" containerName="main"
Sep 12 02:16:06.975146 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:06.975122    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:06.978143 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:06.978120    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-01570e\""
Sep 12 02:16:06.978298 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:06.978161    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-01570e-dockercfg-lg92x\""
Sep 12 02:16:06.984454 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:06.984429    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"]
Sep 12 02:16:07.055880 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.055845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh9pr\" (UniqueName: \"kubernetes.io/projected/4d165f7a-c5c8-4119-83ae-52998fca4e5e-kube-api-access-lh9pr\") pod \"payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm\" (UID: \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:07.056037 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.055934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d165f7a-c5c8-4119-83ae-52998fca4e5e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm\" (UID: \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:07.157146 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.157107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh9pr\" (UniqueName: \"kubernetes.io/projected/4d165f7a-c5c8-4119-83ae-52998fca4e5e-kube-api-access-lh9pr\") pod \"payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm\" (UID: \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:07.157146 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.157158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d165f7a-c5c8-4119-83ae-52998fca4e5e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm\" (UID: \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:07.157866 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.157846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d165f7a-c5c8-4119-83ae-52998fca4e5e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm\" (UID: \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:07.165137 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.165112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh9pr\" (UniqueName: \"kubernetes.io/projected/4d165f7a-c5c8-4119-83ae-52998fca4e5e-kube-api-access-lh9pr\") pod \"payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm\" (UID: \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:07.287506 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.287474    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:07.562077 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.562047    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"]
Sep 12 02:16:07.563029 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:16:07.563008    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d165f7a_c5c8_4119_83ae_52998fca4e5e.slice/crio-d05ba8a60354a931f02546ab61617e3cee5299e3dce6daefbd6d9c2c81d3f875 WatchSource:0}: Error finding container d05ba8a60354a931f02546ab61617e3cee5299e3dce6daefbd6d9c2c81d3f875: Status 404 returned error can't find the container with id d05ba8a60354a931f02546ab61617e3cee5299e3dce6daefbd6d9c2c81d3f875
Sep 12 02:16:07.815093 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.815005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm" event={"ID":"4d165f7a-c5c8-4119-83ae-52998fca4e5e","Type":"ContainerStarted","Data":"753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653"}
Sep 12 02:16:07.815093 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.815060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm" event={"ID":"4d165f7a-c5c8-4119-83ae-52998fca4e5e","Type":"ContainerStarted","Data":"d05ba8a60354a931f02546ab61617e3cee5299e3dce6daefbd6d9c2c81d3f875"}
Sep 12 02:16:07.815319 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.815136    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:07.832447 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:07.832404    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm" podStartSLOduration=1.8323905489999999 podStartE2EDuration="1.832390549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:16:07.831323682 +0000 UTC m=+1873.358224762" watchObservedRunningTime="2026-09-12 02:16:07.832390549 +0000 UTC m=+1873.359291649"
Sep 12 02:16:18.823353 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:18.823281    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:16:34.873455 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:34.873416    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm_4d165f7a-c5c8-4119-83ae-52998fca4e5e/payload-processing/0.log"
Sep 12 02:16:40.741296 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.741242    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"]
Sep 12 02:16:40.745140 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.745123    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.747825 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.747803    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-103dc5\"/\"kube-root-ca.crt\""
Sep 12 02:16:40.748820 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.748795    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-103dc5\"/\"default-dockercfg-w26td\""
Sep 12 02:16:40.748820 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.748812    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-103dc5\"/\"openshift-service-ca.crt\""
Sep 12 02:16:40.748973 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.748883    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-103dc5\"/\"ipp-route-103dc5-kserve-self-signed-certs\""
Sep 12 02:16:40.756054 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.756031    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"]
Sep 12 02:16:40.851734 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.851689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tls-certs\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.851734 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.851740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-dshm\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.851992 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.851849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw4g5\" (UniqueName: \"kubernetes.io/projected/fb9f69a4-2447-4c8f-ab5c-822430572c4f-kube-api-access-xw4g5\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.851992 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.851985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-home\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.852108 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.852021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tmp-dir\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.852108 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.852052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-model-cache\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.952872 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.952822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4g5\" (UniqueName: \"kubernetes.io/projected/fb9f69a4-2447-4c8f-ab5c-822430572c4f-kube-api-access-xw4g5\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.953066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.952942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-home\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.953066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.952963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tmp-dir\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.953066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.952981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-model-cache\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.953066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.953006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tls-certs\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.953066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.953024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-dshm\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.953463 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.953439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tmp-dir\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.953536 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.953475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-home\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.953536 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.953500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-model-cache\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.955292 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.955276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-dshm\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.955455 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.955438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tls-certs\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:40.960748 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:40.960724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw4g5\" (UniqueName: \"kubernetes.io/projected/fb9f69a4-2447-4c8f-ab5c-822430572c4f-kube-api-access-xw4g5\") pod \"ipp-route-103dc5-kserve-74657fb9d7-hzk52\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") " pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:41.054484 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:41.054458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:41.329544 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:41.329514    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"]
Sep 12 02:16:41.330876 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:16:41.330841    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb9f69a4_2447_4c8f_ab5c_822430572c4f.slice/crio-9425dfafa02d62b388af72121caa090147e27b0fce5e8806072303d1598403c0 WatchSource:0}: Error finding container 9425dfafa02d62b388af72121caa090147e27b0fce5e8806072303d1598403c0: Status 404 returned error can't find the container with id 9425dfafa02d62b388af72121caa090147e27b0fce5e8806072303d1598403c0
Sep 12 02:16:42.022537 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:42.022499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52" event={"ID":"fb9f69a4-2447-4c8f-ab5c-822430572c4f","Type":"ContainerStarted","Data":"6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad"}
Sep 12 02:16:42.022537 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:42.022540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52" event={"ID":"fb9f69a4-2447-4c8f-ab5c-822430572c4f","Type":"ContainerStarted","Data":"9425dfafa02d62b388af72121caa090147e27b0fce5e8806072303d1598403c0"}
Sep 12 02:16:42.041603 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:42.041557    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52" podStartSLOduration=2.041542452 podStartE2EDuration="2.041542452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:16:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:16:42.040327787 +0000 UTC m=+1907.567228866" watchObservedRunningTime="2026-09-12 02:16:42.041542452 +0000 UTC m=+1907.568443530"
Sep 12 02:16:51.055540 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:51.055505    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:51.055992 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:51.055554    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:51.068724 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:51.068689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:51.094037 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:51.094015    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:16:55.683452 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:55.683413    2580 scope.go:117] "RemoveContainer" containerID="774b79c2c53c54fb5c386437a6e3c37ce115dc614d63ec21a638fa3c941c6b4c"
Sep 12 02:16:55.693098 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:55.693073    2580 scope.go:117] "RemoveContainer" containerID="5e5eea3b724535eb8e9ab6de64f3870078dd878d001e502b61892fe57b66a78e"
Sep 12 02:16:55.702030 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:16:55.702006    2580 scope.go:117] "RemoveContainer" containerID="1dbcf17164ed31c79d3d53c6d110cb3fbac0375168fdc66b7efa7a5d00a07733"
Sep 12 02:17:10.825127 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:10.825089    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj_036b809b-3e67-492b-b985-4447bb54be9d/payload-processing/0.log"
Sep 12 02:17:11.102761 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:11.102686    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm_4d165f7a-c5c8-4119-83ae-52998fca4e5e/payload-processing/0.log"
Sep 12 02:17:40.349974 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.349936    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"]
Sep 12 02:17:40.353595 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.353574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"
Sep 12 02:17:40.356062 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.356042    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:17:40.356152 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.356046    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:17:40.361228 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.361164    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"]
Sep 12 02:17:40.395634 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.395594    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/310cc13f-9db9-4c70-8d9b-65cbdcbd625b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq\" (UID: \"310cc13f-9db9-4c70-8d9b-65cbdcbd625b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"
Sep 12 02:17:40.496943 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.496908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/310cc13f-9db9-4c70-8d9b-65cbdcbd625b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq\" (UID: \"310cc13f-9db9-4c70-8d9b-65cbdcbd625b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"
Sep 12 02:17:40.497592 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.497569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/310cc13f-9db9-4c70-8d9b-65cbdcbd625b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq\" (UID: \"310cc13f-9db9-4c70-8d9b-65cbdcbd625b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"
Sep 12 02:17:40.666861 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.666773    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"
Sep 12 02:17:40.942629 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:40.942581    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"]
Sep 12 02:17:40.947857 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:17:40.947815    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310cc13f_9db9_4c70_8d9b_65cbdcbd625b.slice/crio-2b7d2be0ca4d6eaae69c0517e624e2da683f7bfcfd004ed0a4ee0aa741429f25 WatchSource:0}: Error finding container 2b7d2be0ca4d6eaae69c0517e624e2da683f7bfcfd004ed0a4ee0aa741429f25: Status 404 returned error can't find the container with id 2b7d2be0ca4d6eaae69c0517e624e2da683f7bfcfd004ed0a4ee0aa741429f25
Sep 12 02:17:41.388931 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:41.388892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq" event={"ID":"310cc13f-9db9-4c70-8d9b-65cbdcbd625b","Type":"ContainerStarted","Data":"289ef7e9cc94dd25a9d0499bb6713652afec646fe0740c246983ed4f2bb06e8a"}
Sep 12 02:17:41.389444 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:41.388953    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq" event={"ID":"310cc13f-9db9-4c70-8d9b-65cbdcbd625b","Type":"ContainerStarted","Data":"2b7d2be0ca4d6eaae69c0517e624e2da683f7bfcfd004ed0a4ee0aa741429f25"}
Sep 12 02:17:41.407034 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:41.406982    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq" podStartSLOduration=1.406966846 podStartE2EDuration="1.406966846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:17:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:17:41.404099072 +0000 UTC m=+1966.931000154" watchObservedRunningTime="2026-09-12 02:17:41.406966846 +0000 UTC m=+1966.933867984"
Sep 12 02:17:42.396649 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:42.396613    2580 generic.go:356] "Generic (PLEG): container finished" podID="310cc13f-9db9-4c70-8d9b-65cbdcbd625b" containerID="289ef7e9cc94dd25a9d0499bb6713652afec646fe0740c246983ed4f2bb06e8a" exitCode=0
Sep 12 02:17:42.397111 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:42.396692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq" event={"ID":"310cc13f-9db9-4c70-8d9b-65cbdcbd625b","Type":"ContainerDied","Data":"289ef7e9cc94dd25a9d0499bb6713652afec646fe0740c246983ed4f2bb06e8a"}
Sep 12 02:17:43.538695 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:43.538671    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"
Sep 12 02:17:43.620445 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:43.620403    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/310cc13f-9db9-4c70-8d9b-65cbdcbd625b-maas-api-service-ca\") pod \"310cc13f-9db9-4c70-8d9b-65cbdcbd625b\" (UID: \"310cc13f-9db9-4c70-8d9b-65cbdcbd625b\") "
Sep 12 02:17:43.620803 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:43.620778    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/310cc13f-9db9-4c70-8d9b-65cbdcbd625b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "310cc13f-9db9-4c70-8d9b-65cbdcbd625b" (UID: "310cc13f-9db9-4c70-8d9b-65cbdcbd625b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:17:43.721718 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:43.721648    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/310cc13f-9db9-4c70-8d9b-65cbdcbd625b-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:17:44.410137 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:44.410106    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"
Sep 12 02:17:44.410343 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:44.410152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq" event={"ID":"310cc13f-9db9-4c70-8d9b-65cbdcbd625b","Type":"ContainerDied","Data":"2b7d2be0ca4d6eaae69c0517e624e2da683f7bfcfd004ed0a4ee0aa741429f25"}
Sep 12 02:17:44.410343 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:44.410227    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b7d2be0ca4d6eaae69c0517e624e2da683f7bfcfd004ed0a4ee0aa741429f25"
Sep 12 02:17:46.679592 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:46.679552    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"]
Sep 12 02:17:46.684598 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:46.684568    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-848a2c-vkgjq"]
Sep 12 02:17:46.981303 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:46.981195    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="310cc13f-9db9-4c70-8d9b-65cbdcbd625b" path="/var/lib/kubelet/pods/310cc13f-9db9-4c70-8d9b-65cbdcbd625b/volumes"
Sep 12 02:17:55.731480 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:55.731435    2580 scope.go:117] "RemoveContainer" containerID="950114ed0619fbffccb069cf30118db51d3a656a5b8f59ff7cb59f60178f058f"
Sep 12 02:17:55.740380 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:55.740358    2580 scope.go:117] "RemoveContainer" containerID="667dc481d8d7b72b1782770714afcd15499c78faecca821b1efd1df723c36c9c"
Sep 12 02:17:55.748645 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:55.748622    2580 scope.go:117] "RemoveContainer" containerID="402ba7633f78cf92300c2eae4366a291c0f3d3e0f1b5a87208b915c827803f4e"
Sep 12 02:17:56.899426 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:56.899388    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"]
Sep 12 02:17:56.899825 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:56.899812    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="310cc13f-9db9-4c70-8d9b-65cbdcbd625b" containerName="revoke-keys"
Sep 12 02:17:56.899825 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:56.899825    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="310cc13f-9db9-4c70-8d9b-65cbdcbd625b" containerName="revoke-keys"
Sep 12 02:17:56.899913 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:56.899885    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="310cc13f-9db9-4c70-8d9b-65cbdcbd625b" containerName="revoke-keys"
Sep 12 02:17:56.904668 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:56.904640    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"
Sep 12 02:17:56.907437 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:56.907409    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:17:56.907563 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:56.907474    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:17:56.909266 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:56.909242    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"]
Sep 12 02:17:56.940516 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:56.940486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/085a2863-0f91-40bf-9e23-41a901b7ef0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k\" (UID: \"085a2863-0f91-40bf-9e23-41a901b7ef0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"
Sep 12 02:17:57.041174 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:57.041139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/085a2863-0f91-40bf-9e23-41a901b7ef0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k\" (UID: \"085a2863-0f91-40bf-9e23-41a901b7ef0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"
Sep 12 02:17:57.041852 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:57.041823    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/085a2863-0f91-40bf-9e23-41a901b7ef0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k\" (UID: \"085a2863-0f91-40bf-9e23-41a901b7ef0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"
Sep 12 02:17:57.218026 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:57.217943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"
Sep 12 02:17:57.489560 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:57.489532    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"]
Sep 12 02:17:57.490985 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:17:57.490963    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod085a2863_0f91_40bf_9e23_41a901b7ef0e.slice/crio-f236bca25d13a64fa45d2e8d7edfc057c0a91e38ee93d5238eee88f788b893bc WatchSource:0}: Error finding container f236bca25d13a64fa45d2e8d7edfc057c0a91e38ee93d5238eee88f788b893bc: Status 404 returned error can't find the container with id f236bca25d13a64fa45d2e8d7edfc057c0a91e38ee93d5238eee88f788b893bc
Sep 12 02:17:58.498278 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:58.498245    2580 generic.go:356] "Generic (PLEG): container finished" podID="085a2863-0f91-40bf-9e23-41a901b7ef0e" containerID="9c7674b1524b93903e24e6cda516dbba96229be6409c312d40f0eba453877b96" exitCode=0
Sep 12 02:17:58.498686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:58.498339    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k" event={"ID":"085a2863-0f91-40bf-9e23-41a901b7ef0e","Type":"ContainerDied","Data":"9c7674b1524b93903e24e6cda516dbba96229be6409c312d40f0eba453877b96"}
Sep 12 02:17:58.498686 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:58.498404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k" event={"ID":"085a2863-0f91-40bf-9e23-41a901b7ef0e","Type":"ContainerStarted","Data":"f236bca25d13a64fa45d2e8d7edfc057c0a91e38ee93d5238eee88f788b893bc"}
Sep 12 02:17:59.640754 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:59.640731    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"
Sep 12 02:17:59.768581 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:59.768486    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/085a2863-0f91-40bf-9e23-41a901b7ef0e-maas-api-service-ca\") pod \"085a2863-0f91-40bf-9e23-41a901b7ef0e\" (UID: \"085a2863-0f91-40bf-9e23-41a901b7ef0e\") "
Sep 12 02:17:59.768872 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:59.768848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/085a2863-0f91-40bf-9e23-41a901b7ef0e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "085a2863-0f91-40bf-9e23-41a901b7ef0e" (UID: "085a2863-0f91-40bf-9e23-41a901b7ef0e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:17:59.869606 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:17:59.869574    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/085a2863-0f91-40bf-9e23-41a901b7ef0e-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:00.511672 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:00.511640    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"
Sep 12 02:18:00.511836 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:00.511669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k" event={"ID":"085a2863-0f91-40bf-9e23-41a901b7ef0e","Type":"ContainerDied","Data":"f236bca25d13a64fa45d2e8d7edfc057c0a91e38ee93d5238eee88f788b893bc"}
Sep 12 02:18:00.511836 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:00.511711    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f236bca25d13a64fa45d2e8d7edfc057c0a91e38ee93d5238eee88f788b893bc"
Sep 12 02:18:12.405854 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.405815    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"]
Sep 12 02:18:12.406310 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.406081    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj" podUID="036b809b-3e67-492b-b985-4447bb54be9d" containerName="payload-processing" containerID="cri-o://f809fc25e850036f3ec8b983019bbb357cda72e99c49dd988bba9488a749fc19" gracePeriod=30
Sep 12 02:18:12.411839 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.411809    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"]
Sep 12 02:18:12.412071 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.412030    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k" podUID="e52a2915-0f73-490f-a1c1-001ecd61da6f" containerName="payload-pre-processing" containerID="cri-o://c6fe45d75da5be874ef335bf444983447dd2a1f5f77cfc8e37ccb19fb534b691" gracePeriod=30
Sep 12 02:18:12.552092 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.552060    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"]
Sep 12 02:18:12.556398 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.556354    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-103dc5-mnq8k"]
Sep 12 02:18:12.594832 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.594801    2580 generic.go:356] "Generic (PLEG): container finished" podID="e52a2915-0f73-490f-a1c1-001ecd61da6f" containerID="c6fe45d75da5be874ef335bf444983447dd2a1f5f77cfc8e37ccb19fb534b691" exitCode=0
Sep 12 02:18:12.595001 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.594870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k" event={"ID":"e52a2915-0f73-490f-a1c1-001ecd61da6f","Type":"ContainerDied","Data":"c6fe45d75da5be874ef335bf444983447dd2a1f5f77cfc8e37ccb19fb534b691"}
Sep 12 02:18:12.597178 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.597146    2580 generic.go:356] "Generic (PLEG): container finished" podID="036b809b-3e67-492b-b985-4447bb54be9d" containerID="f809fc25e850036f3ec8b983019bbb357cda72e99c49dd988bba9488a749fc19" exitCode=0
Sep 12 02:18:12.597332 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.597186    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj" event={"ID":"036b809b-3e67-492b-b985-4447bb54be9d","Type":"ContainerDied","Data":"f809fc25e850036f3ec8b983019bbb357cda72e99c49dd988bba9488a749fc19"}
Sep 12 02:18:12.678096 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.678069    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:18:12.698773 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.698748    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:18:12.782536 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.782491    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/036b809b-3e67-492b-b985-4447bb54be9d-plugins-config-volume\") pod \"036b809b-3e67-492b-b985-4447bb54be9d\" (UID: \"036b809b-3e67-492b-b985-4447bb54be9d\") "
Sep 12 02:18:12.782536 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.782545    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9gqq\" (UniqueName: \"kubernetes.io/projected/e52a2915-0f73-490f-a1c1-001ecd61da6f-kube-api-access-t9gqq\") pod \"e52a2915-0f73-490f-a1c1-001ecd61da6f\" (UID: \"e52a2915-0f73-490f-a1c1-001ecd61da6f\") "
Sep 12 02:18:12.782740 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.782578    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52a2915-0f73-490f-a1c1-001ecd61da6f-plugins-config-volume\") pod \"e52a2915-0f73-490f-a1c1-001ecd61da6f\" (UID: \"e52a2915-0f73-490f-a1c1-001ecd61da6f\") "
Sep 12 02:18:12.782740 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.782676    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/036b809b-3e67-492b-b985-4447bb54be9d-kube-api-access-q6x68\") pod \"036b809b-3e67-492b-b985-4447bb54be9d\" (UID: \"036b809b-3e67-492b-b985-4447bb54be9d\") "
Sep 12 02:18:12.782984 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.782956    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/036b809b-3e67-492b-b985-4447bb54be9d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "036b809b-3e67-492b-b985-4447bb54be9d" (UID: "036b809b-3e67-492b-b985-4447bb54be9d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:18:12.783059 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.782997    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e52a2915-0f73-490f-a1c1-001ecd61da6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e52a2915-0f73-490f-a1c1-001ecd61da6f" (UID: "e52a2915-0f73-490f-a1c1-001ecd61da6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:18:12.784803 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.784782    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e52a2915-0f73-490f-a1c1-001ecd61da6f-kube-api-access-t9gqq" (OuterVolumeSpecName: "kube-api-access-t9gqq") pod "e52a2915-0f73-490f-a1c1-001ecd61da6f" (UID: "e52a2915-0f73-490f-a1c1-001ecd61da6f"). InnerVolumeSpecName "kube-api-access-t9gqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:18:12.784932 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.784913    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/036b809b-3e67-492b-b985-4447bb54be9d-kube-api-access-q6x68" (OuterVolumeSpecName: "kube-api-access-q6x68") pod "036b809b-3e67-492b-b985-4447bb54be9d" (UID: "036b809b-3e67-492b-b985-4447bb54be9d"). InnerVolumeSpecName "kube-api-access-q6x68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:18:12.884323 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.884293    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/036b809b-3e67-492b-b985-4447bb54be9d-kube-api-access-q6x68\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:12.884323 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.884321    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/036b809b-3e67-492b-b985-4447bb54be9d-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:12.884523 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.884333    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9gqq\" (UniqueName: \"kubernetes.io/projected/e52a2915-0f73-490f-a1c1-001ecd61da6f-kube-api-access-t9gqq\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:12.884523 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.884342    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52a2915-0f73-490f-a1c1-001ecd61da6f-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:12.979689 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:12.979616    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="085a2863-0f91-40bf-9e23-41a901b7ef0e" path="/var/lib/kubelet/pods/085a2863-0f91-40bf-9e23-41a901b7ef0e/volumes"
Sep 12 02:18:13.604922 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.604888    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"
Sep 12 02:18:13.604922 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.604919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj" event={"ID":"036b809b-3e67-492b-b985-4447bb54be9d","Type":"ContainerDied","Data":"2a6c0e1aea5c0d785a9c94ef83635e07c719e3baf24d71a182ee33a7f0bd6f17"}
Sep 12 02:18:13.605569 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.604980    2580 scope.go:117] "RemoveContainer" containerID="f809fc25e850036f3ec8b983019bbb357cda72e99c49dd988bba9488a749fc19"
Sep 12 02:18:13.606692 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.606538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k" event={"ID":"e52a2915-0f73-490f-a1c1-001ecd61da6f","Type":"ContainerDied","Data":"5de6a2b4de4e8ec2b29722705a9fd74bb7a9fcc14bc69b885d59270bd17bb70f"}
Sep 12 02:18:13.606692 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.606559    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"
Sep 12 02:18:13.616096 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.615979    2580 scope.go:117] "RemoveContainer" containerID="c6fe45d75da5be874ef335bf444983447dd2a1f5f77cfc8e37ccb19fb534b691"
Sep 12 02:18:13.626960 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.626940    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"]
Sep 12 02:18:13.647370 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.647339    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-103dc5-b7cc5d574-4576k"]
Sep 12 02:18:13.670956 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.670930    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"]
Sep 12 02:18:13.691799 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:13.691737    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-103dc5-65c89df9b9-v2lfj"]
Sep 12 02:18:14.979732 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:14.979699    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="036b809b-3e67-492b-b985-4447bb54be9d" path="/var/lib/kubelet/pods/036b809b-3e67-492b-b985-4447bb54be9d/volumes"
Sep 12 02:18:14.980101 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:14.980067    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e52a2915-0f73-490f-a1c1-001ecd61da6f" path="/var/lib/kubelet/pods/e52a2915-0f73-490f-a1c1-001ecd61da6f/volumes"
Sep 12 02:18:17.940750 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:17.940709    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"]
Sep 12 02:18:32.960065 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:32.959999    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52" podUID="fb9f69a4-2447-4c8f-ab5c-822430572c4f" containerName="main" containerID="cri-o://6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad" gracePeriod=45
Sep 12 02:18:33.216991 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.216931    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:18:33.263909 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.263875    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-dshm\") pod \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") "
Sep 12 02:18:33.264082 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.263943    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-home\") pod \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") "
Sep 12 02:18:33.264082 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.263999    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-model-cache\") pod \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") "
Sep 12 02:18:33.264225 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.264137    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tmp-dir\") pod \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") "
Sep 12 02:18:33.264225 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.264170    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tls-certs\") pod \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") "
Sep 12 02:18:33.264334 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.264229    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw4g5\" (UniqueName: \"kubernetes.io/projected/fb9f69a4-2447-4c8f-ab5c-822430572c4f-kube-api-access-xw4g5\") pod \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\" (UID: \"fb9f69a4-2447-4c8f-ab5c-822430572c4f\") "
Sep 12 02:18:33.264334 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.264284    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-home" (OuterVolumeSpecName: "home") pod "fb9f69a4-2447-4c8f-ab5c-822430572c4f" (UID: "fb9f69a4-2447-4c8f-ab5c-822430572c4f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:18:33.264334 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.264296    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb9f69a4-2447-4c8f-ab5c-822430572c4f" (UID: "fb9f69a4-2447-4c8f-ab5c-822430572c4f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:18:33.264479 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.264424    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb9f69a4-2447-4c8f-ab5c-822430572c4f" (UID: "fb9f69a4-2447-4c8f-ab5c-822430572c4f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:18:33.264607 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.264585    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:33.264674 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.264610    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:33.264674 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.264625    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:33.266490 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.266460    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb9f69a4-2447-4c8f-ab5c-822430572c4f" (UID: "fb9f69a4-2447-4c8f-ab5c-822430572c4f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:18:33.266598 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.266580    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb9f69a4-2447-4c8f-ab5c-822430572c4f-kube-api-access-xw4g5" (OuterVolumeSpecName: "kube-api-access-xw4g5") pod "fb9f69a4-2447-4c8f-ab5c-822430572c4f" (UID: "fb9f69a4-2447-4c8f-ab5c-822430572c4f"). InnerVolumeSpecName "kube-api-access-xw4g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:18:33.266724 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.266703    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-dshm" (OuterVolumeSpecName: "dshm") pod "fb9f69a4-2447-4c8f-ab5c-822430572c4f" (UID: "fb9f69a4-2447-4c8f-ab5c-822430572c4f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:18:33.365593 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.365566    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb9f69a4-2447-4c8f-ab5c-822430572c4f-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:33.365593 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.365591    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw4g5\" (UniqueName: \"kubernetes.io/projected/fb9f69a4-2447-4c8f-ab5c-822430572c4f-kube-api-access-xw4g5\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:33.365593 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.365600    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb9f69a4-2447-4c8f-ab5c-822430572c4f-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:18:33.729616 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.729579    2580 generic.go:356] "Generic (PLEG): container finished" podID="fb9f69a4-2447-4c8f-ab5c-822430572c4f" containerID="6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad" exitCode=0
Sep 12 02:18:33.729788 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.729635    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"
Sep 12 02:18:33.729788 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.729665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52" event={"ID":"fb9f69a4-2447-4c8f-ab5c-822430572c4f","Type":"ContainerDied","Data":"6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad"}
Sep 12 02:18:33.729788 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.729711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52" event={"ID":"fb9f69a4-2447-4c8f-ab5c-822430572c4f","Type":"ContainerDied","Data":"9425dfafa02d62b388af72121caa090147e27b0fce5e8806072303d1598403c0"}
Sep 12 02:18:33.729788 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.729729    2580 scope.go:117] "RemoveContainer" containerID="6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad"
Sep 12 02:18:33.739454 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.739439    2580 scope.go:117] "RemoveContainer" containerID="6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad"
Sep 12 02:18:33.739719 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:18:33.739699    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad\": container with ID starting with 6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad not found: ID does not exist" containerID="6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad"
Sep 12 02:18:33.739775 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.739728    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad"} err="failed to get container status \"6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad\": rpc error: code = NotFound desc = could not find container \"6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad\": container with ID starting with 6cd906d6cf16a578a53720302717cdca3fc9b029a1686d64e36b2e174385afad not found: ID does not exist"
Sep 12 02:18:33.752496 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.752474    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"]
Sep 12 02:18:33.756287 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:33.756266    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-103dc5/ipp-route-103dc5-kserve-74657fb9d7-hzk52"]
Sep 12 02:18:34.980274 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:18:34.980231    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb9f69a4-2447-4c8f-ab5c-822430572c4f" path="/var/lib/kubelet/pods/fb9f69a4-2447-4c8f-ab5c-822430572c4f/volumes"
Sep 12 02:19:15.144258 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144222    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"]
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144577    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb9f69a4-2447-4c8f-ab5c-822430572c4f" containerName="main"
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144588    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb9f69a4-2447-4c8f-ab5c-822430572c4f" containerName="main"
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144596    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="085a2863-0f91-40bf-9e23-41a901b7ef0e" containerName="revoke-keys"
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144601    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="085a2863-0f91-40bf-9e23-41a901b7ef0e" containerName="revoke-keys"
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144613    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="036b809b-3e67-492b-b985-4447bb54be9d" containerName="payload-processing"
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144618    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="036b809b-3e67-492b-b985-4447bb54be9d" containerName="payload-processing"
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144636    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e52a2915-0f73-490f-a1c1-001ecd61da6f" containerName="payload-pre-processing"
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144642    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e52a2915-0f73-490f-a1c1-001ecd61da6f" containerName="payload-pre-processing"
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144692    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e52a2915-0f73-490f-a1c1-001ecd61da6f" containerName="payload-pre-processing"
Sep 12 02:19:15.144696 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144699    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="036b809b-3e67-492b-b985-4447bb54be9d" containerName="payload-processing"
Sep 12 02:19:15.144998 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144708    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="085a2863-0f91-40bf-9e23-41a901b7ef0e" containerName="revoke-keys"
Sep 12 02:19:15.144998 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.144713    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb9f69a4-2447-4c8f-ab5c-822430572c4f" containerName="main"
Sep 12 02:19:15.147885 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.147868    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"
Sep 12 02:19:15.150570 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.150546    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:19:15.150699 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.150574    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:19:15.154930 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.154902    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"]
Sep 12 02:19:15.333666 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.333633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bff8c67-5249-4452-b6aa-7c8be91f9093-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp\" (UID: \"0bff8c67-5249-4452-b6aa-7c8be91f9093\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"
Sep 12 02:19:15.434263 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.434149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bff8c67-5249-4452-b6aa-7c8be91f9093-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp\" (UID: \"0bff8c67-5249-4452-b6aa-7c8be91f9093\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"
Sep 12 02:19:15.434832 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.434806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bff8c67-5249-4452-b6aa-7c8be91f9093-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp\" (UID: \"0bff8c67-5249-4452-b6aa-7c8be91f9093\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"
Sep 12 02:19:15.459281 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.459257    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"
Sep 12 02:19:15.735312 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.735254    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"]
Sep 12 02:19:15.737458 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:19:15.737422    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bff8c67_5249_4452_b6aa_7c8be91f9093.slice/crio-87f105852ce8a3206b36e8ff122d83d9d234e68e20700b6e6c992ca2001619f2 WatchSource:0}: Error finding container 87f105852ce8a3206b36e8ff122d83d9d234e68e20700b6e6c992ca2001619f2: Status 404 returned error can't find the container with id 87f105852ce8a3206b36e8ff122d83d9d234e68e20700b6e6c992ca2001619f2
Sep 12 02:19:15.972412 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:15.972377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp" event={"ID":"0bff8c67-5249-4452-b6aa-7c8be91f9093","Type":"ContainerStarted","Data":"87f105852ce8a3206b36e8ff122d83d9d234e68e20700b6e6c992ca2001619f2"}
Sep 12 02:19:16.980312 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:16.980229    2580 generic.go:356] "Generic (PLEG): container finished" podID="0bff8c67-5249-4452-b6aa-7c8be91f9093" containerID="1d7a61a30fb41ebde35316d97bf66b02457e4153d531c6dad9148e0391c99f92" exitCode=0
Sep 12 02:19:16.980312 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:16.980284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp" event={"ID":"0bff8c67-5249-4452-b6aa-7c8be91f9093","Type":"ContainerDied","Data":"1d7a61a30fb41ebde35316d97bf66b02457e4153d531c6dad9148e0391c99f92"}
Sep 12 02:19:18.117474 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:18.117449    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"
Sep 12 02:19:18.258561 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:18.258533    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bff8c67-5249-4452-b6aa-7c8be91f9093-maas-api-service-ca\") pod \"0bff8c67-5249-4452-b6aa-7c8be91f9093\" (UID: \"0bff8c67-5249-4452-b6aa-7c8be91f9093\") "
Sep 12 02:19:18.258850 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:18.258829    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bff8c67-5249-4452-b6aa-7c8be91f9093-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0bff8c67-5249-4452-b6aa-7c8be91f9093" (UID: "0bff8c67-5249-4452-b6aa-7c8be91f9093"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:19:18.359402 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:18.359366    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bff8c67-5249-4452-b6aa-7c8be91f9093-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:18.992107 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:18.992083    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"
Sep 12 02:19:18.992283 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:18.992114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp" event={"ID":"0bff8c67-5249-4452-b6aa-7c8be91f9093","Type":"ContainerDied","Data":"87f105852ce8a3206b36e8ff122d83d9d234e68e20700b6e6c992ca2001619f2"}
Sep 12 02:19:18.992283 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:18.992155    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87f105852ce8a3206b36e8ff122d83d9d234e68e20700b6e6c992ca2001619f2"
Sep 12 02:19:25.633609 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:25.633563    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"]
Sep 12 02:19:25.634952 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:25.634484    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm" podUID="4d165f7a-c5c8-4119-83ae-52998fca4e5e" containerName="payload-processing" containerID="cri-o://753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653" gracePeriod=30
Sep 12 02:19:25.849845 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:25.849789    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"]
Sep 12 02:19:25.852655 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:25.852631    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-01570e-87cnp"]
Sep 12 02:19:25.901888 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:25.901864    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:19:25.911418 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:25.911395    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh9pr\" (UniqueName: \"kubernetes.io/projected/4d165f7a-c5c8-4119-83ae-52998fca4e5e-kube-api-access-lh9pr\") pod \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\" (UID: \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\") "
Sep 12 02:19:25.911583 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:25.911443    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d165f7a-c5c8-4119-83ae-52998fca4e5e-plugins-config-volume\") pod \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\" (UID: \"4d165f7a-c5c8-4119-83ae-52998fca4e5e\") "
Sep 12 02:19:25.911852 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:25.911819    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d165f7a-c5c8-4119-83ae-52998fca4e5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d165f7a-c5c8-4119-83ae-52998fca4e5e" (UID: "4d165f7a-c5c8-4119-83ae-52998fca4e5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:19:25.913625 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:25.913601    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d165f7a-c5c8-4119-83ae-52998fca4e5e-kube-api-access-lh9pr" (OuterVolumeSpecName: "kube-api-access-lh9pr") pod "4d165f7a-c5c8-4119-83ae-52998fca4e5e" (UID: "4d165f7a-c5c8-4119-83ae-52998fca4e5e"). InnerVolumeSpecName "kube-api-access-lh9pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:19:26.012635 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.012604    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh9pr\" (UniqueName: \"kubernetes.io/projected/4d165f7a-c5c8-4119-83ae-52998fca4e5e-kube-api-access-lh9pr\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:26.012635 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.012633    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d165f7a-c5c8-4119-83ae-52998fca4e5e-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:26.031720 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.031687    2580 generic.go:356] "Generic (PLEG): container finished" podID="4d165f7a-c5c8-4119-83ae-52998fca4e5e" containerID="753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653" exitCode=0
Sep 12 02:19:26.031854 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.031731    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"
Sep 12 02:19:26.031854 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.031734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm" event={"ID":"4d165f7a-c5c8-4119-83ae-52998fca4e5e","Type":"ContainerDied","Data":"753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653"}
Sep 12 02:19:26.031854 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.031791    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm" event={"ID":"4d165f7a-c5c8-4119-83ae-52998fca4e5e","Type":"ContainerDied","Data":"d05ba8a60354a931f02546ab61617e3cee5299e3dce6daefbd6d9c2c81d3f875"}
Sep 12 02:19:26.031854 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.031822    2580 scope.go:117] "RemoveContainer" containerID="753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653"
Sep 12 02:19:26.041232 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.041198    2580 scope.go:117] "RemoveContainer" containerID="753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653"
Sep 12 02:19:26.041510 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:19:26.041491    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653\": container with ID starting with 753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653 not found: ID does not exist" containerID="753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653"
Sep 12 02:19:26.041577 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.041520    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653"} err="failed to get container status \"753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653\": rpc error: code = NotFound desc = could not find container \"753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653\": container with ID starting with 753930404fdef93d6820c73bb1292b536cb98918a37245a65a91d26b9be73653 not found: ID does not exist"
Sep 12 02:19:26.053201 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.053176    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"]
Sep 12 02:19:26.060796 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.060774    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-01570e-6bcd79fb79-jv6jm"]
Sep 12 02:19:26.980056 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.980022    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bff8c67-5249-4452-b6aa-7c8be91f9093" path="/var/lib/kubelet/pods/0bff8c67-5249-4452-b6aa-7c8be91f9093/volumes"
Sep 12 02:19:26.980453 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:26.980427    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d165f7a-c5c8-4119-83ae-52998fca4e5e" path="/var/lib/kubelet/pods/4d165f7a-c5c8-4119-83ae-52998fca4e5e/volumes"
Sep 12 02:19:32.590106 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.590072    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"]
Sep 12 02:19:32.590499 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.590474    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d165f7a-c5c8-4119-83ae-52998fca4e5e" containerName="payload-processing"
Sep 12 02:19:32.590499 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.590487    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d165f7a-c5c8-4119-83ae-52998fca4e5e" containerName="payload-processing"
Sep 12 02:19:32.590577 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.590506    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bff8c67-5249-4452-b6aa-7c8be91f9093" containerName="revoke-keys"
Sep 12 02:19:32.590577 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.590512    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bff8c67-5249-4452-b6aa-7c8be91f9093" containerName="revoke-keys"
Sep 12 02:19:32.590638 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.590577    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d165f7a-c5c8-4119-83ae-52998fca4e5e" containerName="payload-processing"
Sep 12 02:19:32.590638 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.590588    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bff8c67-5249-4452-b6aa-7c8be91f9093" containerName="revoke-keys"
Sep 12 02:19:32.593712 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.593694    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"
Sep 12 02:19:32.596387 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.596356    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:19:32.596387 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.596377    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:19:32.601106 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.601082    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"]
Sep 12 02:19:32.668155 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.668128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8586a3ad-9e60-49bf-8edc-f2d8e42f6c46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj\" (UID: \"8586a3ad-9e60-49bf-8edc-f2d8e42f6c46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"
Sep 12 02:19:32.769637 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.769603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8586a3ad-9e60-49bf-8edc-f2d8e42f6c46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj\" (UID: \"8586a3ad-9e60-49bf-8edc-f2d8e42f6c46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"
Sep 12 02:19:32.770290 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.770264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8586a3ad-9e60-49bf-8edc-f2d8e42f6c46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj\" (UID: \"8586a3ad-9e60-49bf-8edc-f2d8e42f6c46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"
Sep 12 02:19:32.904814 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:32.904731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"
Sep 12 02:19:33.177586 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:33.177555    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"]
Sep 12 02:19:33.179311 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:19:33.179285    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8586a3ad_9e60_49bf_8edc_f2d8e42f6c46.slice/crio-70e1b4c8d38f78e726f1640e34a58401f68cbeba87ee1482900e6c042b47d1b2 WatchSource:0}: Error finding container 70e1b4c8d38f78e726f1640e34a58401f68cbeba87ee1482900e6c042b47d1b2: Status 404 returned error can't find the container with id 70e1b4c8d38f78e726f1640e34a58401f68cbeba87ee1482900e6c042b47d1b2
Sep 12 02:19:34.082631 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:34.082546    2580 generic.go:356] "Generic (PLEG): container finished" podID="8586a3ad-9e60-49bf-8edc-f2d8e42f6c46" containerID="505edce3354dceb73ea134b62615bae8aae4c75a9e2cc7220008d8af122736f9" exitCode=0
Sep 12 02:19:34.082976 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:34.082633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj" event={"ID":"8586a3ad-9e60-49bf-8edc-f2d8e42f6c46","Type":"ContainerDied","Data":"505edce3354dceb73ea134b62615bae8aae4c75a9e2cc7220008d8af122736f9"}
Sep 12 02:19:34.082976 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:34.082684    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj" event={"ID":"8586a3ad-9e60-49bf-8edc-f2d8e42f6c46","Type":"ContainerStarted","Data":"70e1b4c8d38f78e726f1640e34a58401f68cbeba87ee1482900e6c042b47d1b2"}
Sep 12 02:19:35.217365 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:35.217339    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"
Sep 12 02:19:35.293424 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:35.293393    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8586a3ad-9e60-49bf-8edc-f2d8e42f6c46-maas-api-service-ca\") pod \"8586a3ad-9e60-49bf-8edc-f2d8e42f6c46\" (UID: \"8586a3ad-9e60-49bf-8edc-f2d8e42f6c46\") "
Sep 12 02:19:35.293744 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:35.293722    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8586a3ad-9e60-49bf-8edc-f2d8e42f6c46-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8586a3ad-9e60-49bf-8edc-f2d8e42f6c46" (UID: "8586a3ad-9e60-49bf-8edc-f2d8e42f6c46"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:19:35.394300 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:35.394206    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8586a3ad-9e60-49bf-8edc-f2d8e42f6c46-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:36.096512 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:36.096473    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"
Sep 12 02:19:36.096685 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:36.096508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj" event={"ID":"8586a3ad-9e60-49bf-8edc-f2d8e42f6c46","Type":"ContainerDied","Data":"70e1b4c8d38f78e726f1640e34a58401f68cbeba87ee1482900e6c042b47d1b2"}
Sep 12 02:19:36.096685 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:36.096555    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70e1b4c8d38f78e726f1640e34a58401f68cbeba87ee1482900e6c042b47d1b2"
Sep 12 02:19:43.035729 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.035696    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"]
Sep 12 02:19:43.036130 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.035926    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762" podUID="19ce1f88-2488-4f94-b22c-51422a8dda16" containerName="payload-processing" containerID="cri-o://27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580" gracePeriod=30
Sep 12 02:19:43.185060 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.185025    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"]
Sep 12 02:19:43.187413 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.187391    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a744d5-z46rj"]
Sep 12 02:19:43.299665 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.299603    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:19:43.354630 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.354603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-676z4\" (UniqueName: \"kubernetes.io/projected/19ce1f88-2488-4f94-b22c-51422a8dda16-kube-api-access-676z4\") pod \"19ce1f88-2488-4f94-b22c-51422a8dda16\" (UID: \"19ce1f88-2488-4f94-b22c-51422a8dda16\") "
Sep 12 02:19:43.354630 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.354635    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ce1f88-2488-4f94-b22c-51422a8dda16-plugins-config-volume\") pod \"19ce1f88-2488-4f94-b22c-51422a8dda16\" (UID: \"19ce1f88-2488-4f94-b22c-51422a8dda16\") "
Sep 12 02:19:43.355017 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.354991    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19ce1f88-2488-4f94-b22c-51422a8dda16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19ce1f88-2488-4f94-b22c-51422a8dda16" (UID: "19ce1f88-2488-4f94-b22c-51422a8dda16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:19:43.356652 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.356632    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19ce1f88-2488-4f94-b22c-51422a8dda16-kube-api-access-676z4" (OuterVolumeSpecName: "kube-api-access-676z4") pod "19ce1f88-2488-4f94-b22c-51422a8dda16" (UID: "19ce1f88-2488-4f94-b22c-51422a8dda16"). InnerVolumeSpecName "kube-api-access-676z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:19:43.455916 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.455885    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-676z4\" (UniqueName: \"kubernetes.io/projected/19ce1f88-2488-4f94-b22c-51422a8dda16-kube-api-access-676z4\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:43.455916 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:43.455911    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ce1f88-2488-4f94-b22c-51422a8dda16-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:44.143885 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.143846    2580 generic.go:356] "Generic (PLEG): container finished" podID="19ce1f88-2488-4f94-b22c-51422a8dda16" containerID="27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580" exitCode=0
Sep 12 02:19:44.144348 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.143905    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"
Sep 12 02:19:44.144348 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.143939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762" event={"ID":"19ce1f88-2488-4f94-b22c-51422a8dda16","Type":"ContainerDied","Data":"27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580"}
Sep 12 02:19:44.144348 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.144004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762" event={"ID":"19ce1f88-2488-4f94-b22c-51422a8dda16","Type":"ContainerDied","Data":"53f059a1a448ccd6a2306284ebafc6b91ed673b37b4cdf1615ad975be345c2af"}
Sep 12 02:19:44.144348 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.144038    2580 scope.go:117] "RemoveContainer" containerID="27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580"
Sep 12 02:19:44.154171 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.154154    2580 scope.go:117] "RemoveContainer" containerID="27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580"
Sep 12 02:19:44.154449 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:19:44.154429    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580\": container with ID starting with 27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580 not found: ID does not exist" containerID="27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580"
Sep 12 02:19:44.154518 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.154456    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580"} err="failed to get container status \"27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580\": rpc error: code = NotFound desc = could not find container \"27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580\": container with ID starting with 27573e1c595681765997d98a7d1a96480dbf4ae48b9c942ca8eb519ead4c2580 not found: ID does not exist"
Sep 12 02:19:44.168709 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.168685    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"]
Sep 12 02:19:44.175532 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.175509    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-a744d5-d4cbf6d85-fk762"]
Sep 12 02:19:44.980342 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.980302    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19ce1f88-2488-4f94-b22c-51422a8dda16" path="/var/lib/kubelet/pods/19ce1f88-2488-4f94-b22c-51422a8dda16/volumes"
Sep 12 02:19:44.980752 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:44.980728    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8586a3ad-9e60-49bf-8edc-f2d8e42f6c46" path="/var/lib/kubelet/pods/8586a3ad-9e60-49bf-8edc-f2d8e42f6c46/volumes"
Sep 12 02:19:49.809167 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:49.809134    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"]
Sep 12 02:19:49.809653 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:49.809433    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy" containerID="cri-o://9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db" gracePeriod=30
Sep 12 02:19:50.180528 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.180451    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"]
Sep 12 02:19:50.180869 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.180846    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19ce1f88-2488-4f94-b22c-51422a8dda16" containerName="payload-processing"
Sep 12 02:19:50.180869 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.180864    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="19ce1f88-2488-4f94-b22c-51422a8dda16" containerName="payload-processing"
Sep 12 02:19:50.180967 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.180891    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8586a3ad-9e60-49bf-8edc-f2d8e42f6c46" containerName="revoke-keys"
Sep 12 02:19:50.180967 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.180899    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8586a3ad-9e60-49bf-8edc-f2d8e42f6c46" containerName="revoke-keys"
Sep 12 02:19:50.180967 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.180963    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8586a3ad-9e60-49bf-8edc-f2d8e42f6c46" containerName="revoke-keys"
Sep 12 02:19:50.181100 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.180974    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="19ce1f88-2488-4f94-b22c-51422a8dda16" containerName="payload-processing"
Sep 12 02:19:50.185494 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.185475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"
Sep 12 02:19:50.188269 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.188241    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7x88s\""
Sep 12 02:19:50.188398 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.188300    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 02:19:50.190450 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.190425    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"]
Sep 12 02:19:50.319836 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.319804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffef273-e03f-4f9d-9244-57d5f7dbe946-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq\" (UID: \"1ffef273-e03f-4f9d-9244-57d5f7dbe946\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"
Sep 12 02:19:50.421049 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.421021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffef273-e03f-4f9d-9244-57d5f7dbe946-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq\" (UID: \"1ffef273-e03f-4f9d-9244-57d5f7dbe946\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"
Sep 12 02:19:50.421922 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.421893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffef273-e03f-4f9d-9244-57d5f7dbe946-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq\" (UID: \"1ffef273-e03f-4f9d-9244-57d5f7dbe946\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"
Sep 12 02:19:50.497083 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.497055    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"
Sep 12 02:19:50.768529 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:50.768492    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"]
Sep 12 02:19:50.770293 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:19:50.770247    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffef273_e03f_4f9d_9244_57d5f7dbe946.slice/crio-91223fd291ffba6e389c07ee421941af620d813127bc80020220f05e6611dd44 WatchSource:0}: Error finding container 91223fd291ffba6e389c07ee421941af620d813127bc80020220f05e6611dd44: Status 404 returned error can't find the container with id 91223fd291ffba6e389c07ee421941af620d813127bc80020220f05e6611dd44
Sep 12 02:19:51.187519 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:51.187461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq" event={"ID":"1ffef273-e03f-4f9d-9244-57d5f7dbe946","Type":"ContainerStarted","Data":"6a82de88712ad66a556f363dbac2a014fbc7b9f71958d90f6f14b6879218f47b"}
Sep 12 02:19:51.187857 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:51.187543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq" event={"ID":"1ffef273-e03f-4f9d-9244-57d5f7dbe946","Type":"ContainerStarted","Data":"91223fd291ffba6e389c07ee421941af620d813127bc80020220f05e6611dd44"}
Sep 12 02:19:51.202082 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:51.202034    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq" podStartSLOduration=1.202018746 podStartE2EDuration="1.202018746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:19:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:19:51.201510018 +0000 UTC m=+2096.728411092" watchObservedRunningTime="2026-09-12 02:19:51.202018746 +0000 UTC m=+2096.728919825"
Sep 12 02:19:52.195089 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:52.195052    2580 generic.go:356] "Generic (PLEG): container finished" podID="1ffef273-e03f-4f9d-9244-57d5f7dbe946" containerID="6a82de88712ad66a556f363dbac2a014fbc7b9f71958d90f6f14b6879218f47b" exitCode=0
Sep 12 02:19:52.195482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:52.195108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq" event={"ID":"1ffef273-e03f-4f9d-9244-57d5f7dbe946","Type":"ContainerDied","Data":"6a82de88712ad66a556f363dbac2a014fbc7b9f71958d90f6f14b6879218f47b"}
Sep 12 02:19:53.329876 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:53.329852    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"
Sep 12 02:19:53.447273 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:53.447241    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffef273-e03f-4f9d-9244-57d5f7dbe946-maas-api-service-ca\") pod \"1ffef273-e03f-4f9d-9244-57d5f7dbe946\" (UID: \"1ffef273-e03f-4f9d-9244-57d5f7dbe946\") "
Sep 12 02:19:53.447626 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:53.447600    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ffef273-e03f-4f9d-9244-57d5f7dbe946-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1ffef273-e03f-4f9d-9244-57d5f7dbe946" (UID: "1ffef273-e03f-4f9d-9244-57d5f7dbe946"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:19:53.548691 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:53.548664    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ffef273-e03f-4f9d-9244-57d5f7dbe946-maas-api-service-ca\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:54.207806 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:54.207771    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"
Sep 12 02:19:54.207806 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:54.207797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq" event={"ID":"1ffef273-e03f-4f9d-9244-57d5f7dbe946","Type":"ContainerDied","Data":"91223fd291ffba6e389c07ee421941af620d813127bc80020220f05e6611dd44"}
Sep 12 02:19:54.208006 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:54.207848    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91223fd291ffba6e389c07ee421941af620d813127bc80020220f05e6611dd44"
Sep 12 02:19:55.060011 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.059988    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:19:55.160555 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160461    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7njvk\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-kube-api-access-7njvk\") pod \"24a215c5-6700-4b72-a14e-bd1d515d7161\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") "
Sep 12 02:19:55.160555 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-envoy\") pod \"24a215c5-6700-4b72-a14e-bd1d515d7161\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") "
Sep 12 02:19:55.160555 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160535    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-podinfo\") pod \"24a215c5-6700-4b72-a14e-bd1d515d7161\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") "
Sep 12 02:19:55.160810 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160597    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-data\") pod \"24a215c5-6700-4b72-a14e-bd1d515d7161\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") "
Sep 12 02:19:55.160810 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160632    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-certs\") pod \"24a215c5-6700-4b72-a14e-bd1d515d7161\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") "
Sep 12 02:19:55.160810 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160658    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-credential-socket\") pod \"24a215c5-6700-4b72-a14e-bd1d515d7161\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") "
Sep 12 02:19:55.160810 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160727    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-token\") pod \"24a215c5-6700-4b72-a14e-bd1d515d7161\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") "
Sep 12 02:19:55.160810 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160756    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24a215c5-6700-4b72-a14e-bd1d515d7161-istiod-ca-cert\") pod \"24a215c5-6700-4b72-a14e-bd1d515d7161\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") "
Sep 12 02:19:55.160810 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-socket\") pod \"24a215c5-6700-4b72-a14e-bd1d515d7161\" (UID: \"24a215c5-6700-4b72-a14e-bd1d515d7161\") "
Sep 12 02:19:55.161102 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160920    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-data" (OuterVolumeSpecName: "istio-data") pod "24a215c5-6700-4b72-a14e-bd1d515d7161" (UID: "24a215c5-6700-4b72-a14e-bd1d515d7161"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:19:55.161102 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160973    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24a215c5-6700-4b72-a14e-bd1d515d7161" (UID: "24a215c5-6700-4b72-a14e-bd1d515d7161"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:19:55.161102 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.160928    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24a215c5-6700-4b72-a14e-bd1d515d7161" (UID: "24a215c5-6700-4b72-a14e-bd1d515d7161"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:19:55.161319 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.161134    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-data\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:55.161319 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.161155    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:55.161319 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.161154    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24a215c5-6700-4b72-a14e-bd1d515d7161" (UID: "24a215c5-6700-4b72-a14e-bd1d515d7161"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:19:55.161319 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.161170    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-credential-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:55.161319 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.161251    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24a215c5-6700-4b72-a14e-bd1d515d7161-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24a215c5-6700-4b72-a14e-bd1d515d7161" (UID: "24a215c5-6700-4b72-a14e-bd1d515d7161"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:19:55.162772 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.162748    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24a215c5-6700-4b72-a14e-bd1d515d7161" (UID: "24a215c5-6700-4b72-a14e-bd1d515d7161"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:19:55.162772 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.162763    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24a215c5-6700-4b72-a14e-bd1d515d7161" (UID: "24a215c5-6700-4b72-a14e-bd1d515d7161"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 12 02:19:55.162924 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.162870    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-token" (OuterVolumeSpecName: "istio-token") pod "24a215c5-6700-4b72-a14e-bd1d515d7161" (UID: "24a215c5-6700-4b72-a14e-bd1d515d7161"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:19:55.162969 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.162939    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-kube-api-access-7njvk" (OuterVolumeSpecName: "kube-api-access-7njvk") pod "24a215c5-6700-4b72-a14e-bd1d515d7161" (UID: "24a215c5-6700-4b72-a14e-bd1d515d7161"). InnerVolumeSpecName "kube-api-access-7njvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:19:55.204038 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.204018    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:19:55.215222 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.215179    2580 generic.go:356] "Generic (PLEG): container finished" podID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerID="9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db" exitCode=0
Sep 12 02:19:55.215331 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.215218    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" event={"ID":"24a215c5-6700-4b72-a14e-bd1d515d7161","Type":"ContainerDied","Data":"9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db"}
Sep 12 02:19:55.215331 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.215246    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"
Sep 12 02:19:55.215331 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.215278    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm" event={"ID":"24a215c5-6700-4b72-a14e-bd1d515d7161","Type":"ContainerDied","Data":"e27045dc1e890d32e1343bf25f8d59d4527848c2a5c0d26e0931a6a51688f99a"}
Sep 12 02:19:55.215331 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.215309    2580 scope.go:117] "RemoveContainer" containerID="9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db"
Sep 12 02:19:55.221691 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.221672    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:19:55.226188 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.226173    2580 scope.go:117] "RemoveContainer" containerID="674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45"
Sep 12 02:19:55.235018 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.235003    2580 scope.go:117] "RemoveContainer" containerID="9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db"
Sep 12 02:19:55.235258 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:19:55.235241    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db\": container with ID starting with 9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db not found: ID does not exist" containerID="9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db"
Sep 12 02:19:55.235303 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.235265    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db"} err="failed to get container status \"9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db\": rpc error: code = NotFound desc = could not find container \"9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db\": container with ID starting with 9651334b636ab5fb809c53387954038a457f3b5bbf2b9ac74decb1db01d2a4db not found: ID does not exist"
Sep 12 02:19:55.235303 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.235280    2580 scope.go:117] "RemoveContainer" containerID="674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45"
Sep 12 02:19:55.235512 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:19:55.235494    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45\": container with ID starting with 674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45 not found: ID does not exist" containerID="674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45"
Sep 12 02:19:55.235571 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.235522    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45"} err="failed to get container status \"674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45\": rpc error: code = NotFound desc = could not find container \"674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45\": container with ID starting with 674611c1dfefcaeb484a6e4a42010ed3f7a5d1f2ad23b97f369f965401b92a45 not found: ID does not exist"
Sep 12 02:19:55.239150 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.239129    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"]
Sep 12 02:19:55.242914 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.242893    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-a744d5-openshift-default-7d9b86d6f4-vjwcm"]
Sep 12 02:19:55.262488 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.262470    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-workload-socket\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:55.262566 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.262490    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7njvk\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-kube-api-access-7njvk\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:55.262566 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.262501    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-envoy\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:55.262566 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.262510    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-podinfo\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:55.262566 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.262518    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24a215c5-6700-4b72-a14e-bd1d515d7161-istio-token\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:55.262566 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:55.262526    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24a215c5-6700-4b72-a14e-bd1d515d7161-istiod-ca-cert\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:19:56.986043 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:19:56.986003    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" path="/var/lib/kubelet/pods/24a215c5-6700-4b72-a14e-bd1d515d7161/volumes"
Sep 12 02:20:00.714144 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:00.714103    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn"]
Sep 12 02:20:00.716762 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:00.716731    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fbb6d9-29819655-nfjtn"]
Sep 12 02:20:00.737584 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:00.737553    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"]
Sep 12 02:20:00.737849 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:00.737816    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr" podUID="dfac920a-6df2-443c-af07-8cecee6b4d84" containerName="payload-processing" containerID="cri-o://7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee" gracePeriod=30
Sep 12 02:20:00.880279 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:00.880247    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"]
Sep 12 02:20:00.882589 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:00.882567    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fbb6d9-cb7dq"]
Sep 12 02:20:00.981137 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:00.981105    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ffef273-e03f-4f9d-9244-57d5f7dbe946" path="/var/lib/kubelet/pods/1ffef273-e03f-4f9d-9244-57d5f7dbe946/volumes"
Sep 12 02:20:00.981582 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:00.981559    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62f3e447-6326-41c3-b0ab-a30a37ec53a7" path="/var/lib/kubelet/pods/62f3e447-6326-41c3-b0ab-a30a37ec53a7/volumes"
Sep 12 02:20:00.999252 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:00.999218    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:20:01.109002 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.108964    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79stg\" (UniqueName: \"kubernetes.io/projected/dfac920a-6df2-443c-af07-8cecee6b4d84-kube-api-access-79stg\") pod \"dfac920a-6df2-443c-af07-8cecee6b4d84\" (UID: \"dfac920a-6df2-443c-af07-8cecee6b4d84\") "
Sep 12 02:20:01.109192 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.109087    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfac920a-6df2-443c-af07-8cecee6b4d84-plugins-config-volume\") pod \"dfac920a-6df2-443c-af07-8cecee6b4d84\" (UID: \"dfac920a-6df2-443c-af07-8cecee6b4d84\") "
Sep 12 02:20:01.109484 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.109459    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfac920a-6df2-443c-af07-8cecee6b4d84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dfac920a-6df2-443c-af07-8cecee6b4d84" (UID: "dfac920a-6df2-443c-af07-8cecee6b4d84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 02:20:01.111082 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.111057    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfac920a-6df2-443c-af07-8cecee6b4d84-kube-api-access-79stg" (OuterVolumeSpecName: "kube-api-access-79stg") pod "dfac920a-6df2-443c-af07-8cecee6b4d84" (UID: "dfac920a-6df2-443c-af07-8cecee6b4d84"). InnerVolumeSpecName "kube-api-access-79stg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:20:01.210704 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.210660    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79stg\" (UniqueName: \"kubernetes.io/projected/dfac920a-6df2-443c-af07-8cecee6b4d84-kube-api-access-79stg\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:20:01.210704 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.210696    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfac920a-6df2-443c-af07-8cecee6b4d84-plugins-config-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:20:01.253425 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.253389    2580 generic.go:356] "Generic (PLEG): container finished" podID="dfac920a-6df2-443c-af07-8cecee6b4d84" containerID="7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee" exitCode=0
Sep 12 02:20:01.253615 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.253444    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"
Sep 12 02:20:01.253615 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.253475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr" event={"ID":"dfac920a-6df2-443c-af07-8cecee6b4d84","Type":"ContainerDied","Data":"7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee"}
Sep 12 02:20:01.253615 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.253555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr" event={"ID":"dfac920a-6df2-443c-af07-8cecee6b4d84","Type":"ContainerDied","Data":"7e660722bb1df2744b36867ba5c8a3bba3a3fb85e58d0e46b163cb6fd32011e4"}
Sep 12 02:20:01.253615 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.253589    2580 scope.go:117] "RemoveContainer" containerID="7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee"
Sep 12 02:20:01.263593 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.263577    2580 scope.go:117] "RemoveContainer" containerID="7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee"
Sep 12 02:20:01.263880 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:20:01.263862    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee\": container with ID starting with 7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee not found: ID does not exist" containerID="7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee"
Sep 12 02:20:01.263943 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.263889    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee"} err="failed to get container status \"7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee\": rpc error: code = NotFound desc = could not find container \"7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee\": container with ID starting with 7762dedd27b772327a7a6aa3816b9d2cc447083219b7d06815a09d80c4b539ee not found: ID does not exist"
Sep 12 02:20:01.280294 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.280263    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"]
Sep 12 02:20:01.282828 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:01.282809    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fbb6d9-7bb6b758ff-ghwlr"]
Sep 12 02:20:02.980785 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:20:02.980752    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfac920a-6df2-443c-af07-8cecee6b4d84" path="/var/lib/kubelet/pods/dfac920a-6df2-443c-af07-8cecee6b4d84/volumes"
Sep 12 02:21:55.839480 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:21:55.839446    2580 scope.go:117] "RemoveContainer" containerID="2e92bc0526b725fb07671461661791cab16430466c8b0a2591d42dab935fa7cb"
Sep 12 02:22:52.522105 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:52.522069    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"]
Sep 12 02:22:52.522628 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:52.522315    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc" podUID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" containerName="manager" containerID="cri-o://6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992" gracePeriod=10
Sep 12 02:22:52.775795 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:52.775738    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 02:22:52.869373 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:52.869338    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-extensions-socket-volume\") pod \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\" (UID: \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\") "
Sep 12 02:22:52.869541 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:52.869416    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsl29\" (UniqueName: \"kubernetes.io/projected/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-kube-api-access-jsl29\") pod \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\" (UID: \"5f915fcb-6af0-4e0e-9294-deb5f7c772dd\") "
Sep 12 02:22:52.869809 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:52.869778    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5f915fcb-6af0-4e0e-9294-deb5f7c772dd" (UID: "5f915fcb-6af0-4e0e-9294-deb5f7c772dd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:22:52.871521 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:52.871486    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-kube-api-access-jsl29" (OuterVolumeSpecName: "kube-api-access-jsl29") pod "5f915fcb-6af0-4e0e-9294-deb5f7c772dd" (UID: "5f915fcb-6af0-4e0e-9294-deb5f7c772dd"). InnerVolumeSpecName "kube-api-access-jsl29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:22:52.970073 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:52.970036    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsl29\" (UniqueName: \"kubernetes.io/projected/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-kube-api-access-jsl29\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:22:52.970073 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:52.970069    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f915fcb-6af0-4e0e-9294-deb5f7c772dd-extensions-socket-volume\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:22:53.251365 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.251323    2580 generic.go:356] "Generic (PLEG): container finished" podID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" containerID="6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992" exitCode=0
Sep 12 02:22:53.251530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.251375    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"
Sep 12 02:22:53.251530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.251403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc" event={"ID":"5f915fcb-6af0-4e0e-9294-deb5f7c772dd","Type":"ContainerDied","Data":"6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992"}
Sep 12 02:22:53.251530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.251459    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc" event={"ID":"5f915fcb-6af0-4e0e-9294-deb5f7c772dd","Type":"ContainerDied","Data":"2a1647c55def7fa93438a89a59780827c1deee03d5339a0110ba8f8e8cade1e7"}
Sep 12 02:22:53.251530 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.251480    2580 scope.go:117] "RemoveContainer" containerID="6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992"
Sep 12 02:22:53.260807 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.260788    2580 scope.go:117] "RemoveContainer" containerID="90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9"
Sep 12 02:22:53.269424 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.269403    2580 scope.go:117] "RemoveContainer" containerID="6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992"
Sep 12 02:22:53.269761 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:22:53.269739    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992\": container with ID starting with 6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992 not found: ID does not exist" containerID="6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992"
Sep 12 02:22:53.269816 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.269770    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992"} err="failed to get container status \"6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992\": rpc error: code = NotFound desc = could not find container \"6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992\": container with ID starting with 6832507ba8bd796d70aeb18f97468ab20743e9dde7f51a7a7f4b506fde90b992 not found: ID does not exist"
Sep 12 02:22:53.269816 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.269792    2580 scope.go:117] "RemoveContainer" containerID="90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9"
Sep 12 02:22:53.270081 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:22:53.270063    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9\": container with ID starting with 90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9 not found: ID does not exist" containerID="90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9"
Sep 12 02:22:53.270138 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.270084    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9"} err="failed to get container status \"90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9\": rpc error: code = NotFound desc = could not find container \"90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9\": container with ID starting with 90f8001aa30734ed6a440ac8696b4d040d59a51a731ee06eb94aadc1ae98aaf9 not found: ID does not exist"
Sep 12 02:22:53.271096 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.271070    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"]
Sep 12 02:22:53.273802 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:53.273780    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zz6kc"]
Sep 12 02:22:54.978705 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:22:54.978668    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" path="/var/lib/kubelet/pods/5f915fcb-6af0-4e0e-9294-deb5f7c772dd/volumes"
Sep 12 02:23:53.222533 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222494    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"]
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222843    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ffef273-e03f-4f9d-9244-57d5f7dbe946" containerName="revoke-keys"
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222855    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ffef273-e03f-4f9d-9244-57d5f7dbe946" containerName="revoke-keys"
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222864    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" containerName="manager"
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222870    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" containerName="manager"
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222877    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy"
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222911    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy"
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222924    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" containerName="manager"
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222930    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" containerName="manager"
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222938    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfac920a-6df2-443c-af07-8cecee6b4d84" containerName="payload-processing"
Sep 12 02:23:53.222971 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222943    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfac920a-6df2-443c-af07-8cecee6b4d84" containerName="payload-processing"
Sep 12 02:23:53.223360 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.222996    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy"
Sep 12 02:23:53.223360 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.223007    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfac920a-6df2-443c-af07-8cecee6b4d84" containerName="payload-processing"
Sep 12 02:23:53.223360 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.223013    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" containerName="manager"
Sep 12 02:23:53.223360 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.223019    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f915fcb-6af0-4e0e-9294-deb5f7c772dd" containerName="manager"
Sep 12 02:23:53.223360 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.223027    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ffef273-e03f-4f9d-9244-57d5f7dbe946" containerName="revoke-keys"
Sep 12 02:23:53.226065 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.226046    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:23:53.228981 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.228956    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-snrcf\""
Sep 12 02:23:53.236672 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.236643    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"]
Sep 12 02:23:53.290091 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.290051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47n9j\" (UniqueName: \"kubernetes.io/projected/61cda422-13a9-4648-884d-09f60d8ab85d-kube-api-access-47n9j\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6srjq\" (UID: \"61cda422-13a9-4648-884d-09f60d8ab85d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:23:53.290291 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.290094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61cda422-13a9-4648-884d-09f60d8ab85d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6srjq\" (UID: \"61cda422-13a9-4648-884d-09f60d8ab85d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:23:53.391577 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.391530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47n9j\" (UniqueName: \"kubernetes.io/projected/61cda422-13a9-4648-884d-09f60d8ab85d-kube-api-access-47n9j\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6srjq\" (UID: \"61cda422-13a9-4648-884d-09f60d8ab85d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:23:53.391766 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.391585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61cda422-13a9-4648-884d-09f60d8ab85d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6srjq\" (UID: \"61cda422-13a9-4648-884d-09f60d8ab85d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:23:53.391988 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.391966    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61cda422-13a9-4648-884d-09f60d8ab85d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6srjq\" (UID: \"61cda422-13a9-4648-884d-09f60d8ab85d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:23:53.399721 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.399694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47n9j\" (UniqueName: \"kubernetes.io/projected/61cda422-13a9-4648-884d-09f60d8ab85d-kube-api-access-47n9j\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6srjq\" (UID: \"61cda422-13a9-4648-884d-09f60d8ab85d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:23:53.538092 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.538057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:23:53.832581 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:53.832553    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"]
Sep 12 02:23:53.833553 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:23:53.833526    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61cda422_13a9_4648_884d_09f60d8ab85d.slice/crio-e61e0cb0b2517f248a821ca444d31b4738bc59d6f7a4ff7b76a3731e40fb82c8 WatchSource:0}: Error finding container e61e0cb0b2517f248a821ca444d31b4738bc59d6f7a4ff7b76a3731e40fb82c8: Status 404 returned error can't find the container with id e61e0cb0b2517f248a821ca444d31b4738bc59d6f7a4ff7b76a3731e40fb82c8
Sep 12 02:23:54.591074 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:54.591032    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq" event={"ID":"61cda422-13a9-4648-884d-09f60d8ab85d","Type":"ContainerStarted","Data":"c9f8b45290dfbfad17c05e793e206639e58fbe5f9e0cfce915560952a3779284"}
Sep 12 02:23:54.591498 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:54.591233    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:23:54.591498 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:54.591279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq" event={"ID":"61cda422-13a9-4648-884d-09f60d8ab85d","Type":"ContainerStarted","Data":"e61e0cb0b2517f248a821ca444d31b4738bc59d6f7a4ff7b76a3731e40fb82c8"}
Sep 12 02:23:54.612002 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:54.611948    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq" podStartSLOduration=1.611935034 podStartE2EDuration="1.611935034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:23:54.609120594 +0000 UTC m=+2340.136021674" watchObservedRunningTime="2026-09-12 02:23:54.611935034 +0000 UTC m=+2340.138836113"
Sep 12 02:23:55.879625 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:23:55.879589    2580 scope.go:117] "RemoveContainer" containerID="289ef7e9cc94dd25a9d0499bb6713652afec646fe0740c246983ed4f2bb06e8a"
Sep 12 02:24:05.599270 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:05.599231    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6srjq"
Sep 12 02:24:42.723915 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.723880    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"]
Sep 12 02:24:42.724552 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.724250    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy"
Sep 12 02:24:42.724552 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.724264    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy"
Sep 12 02:24:42.724552 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.724344    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="24a215c5-6700-4b72-a14e-bd1d515d7161" containerName="istio-proxy"
Sep 12 02:24:42.727393 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.727368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.729996 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.729969    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ecd0309c-kserve-self-signed-certs\""
Sep 12 02:24:42.738039 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.737979    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"]
Sep 12 02:24:42.832482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.832436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tls-certs\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.832690 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.832501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-model-cache\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.832690 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.832590    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qww9m\" (UniqueName: \"kubernetes.io/projected/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-kube-api-access-qww9m\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.832690 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.832626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-home\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.832690 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.832682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tmp-dir\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.832887 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.832751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-dshm\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.933794 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.933759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-dshm\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.933985 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.933805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tls-certs\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.933985 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.933849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-model-cache\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.934118 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.933990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qww9m\" (UniqueName: \"kubernetes.io/projected/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-kube-api-access-qww9m\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.934118 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.934048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-home\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.934118 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.934094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tmp-dir\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.934312 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.934269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-model-cache\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.934430 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.934404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-home\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.934498 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.934460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tmp-dir\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.936051 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.936027    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-dshm\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.936332 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.936315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tls-certs\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:42.942355 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:42.942333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qww9m\" (UniqueName: \"kubernetes.io/projected/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-kube-api-access-qww9m\") pod \"e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") " pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:43.040157 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:43.040126    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:43.312859 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:43.312807    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"]
Sep 12 02:24:43.314456 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:24:43.314430    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8fe96f3_ac40_46f6_b1fe_3aaa5ef567b7.slice/crio-237d488581a487846e2e2d05241d94daf7efd44a13255b6fbc167e7f6d0ea2a6 WatchSource:0}: Error finding container 237d488581a487846e2e2d05241d94daf7efd44a13255b6fbc167e7f6d0ea2a6: Status 404 returned error can't find the container with id 237d488581a487846e2e2d05241d94daf7efd44a13255b6fbc167e7f6d0ea2a6
Sep 12 02:24:43.884024 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:43.883979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724" event={"ID":"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7","Type":"ContainerStarted","Data":"3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b"}
Sep 12 02:24:43.884505 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:43.884039    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724" event={"ID":"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7","Type":"ContainerStarted","Data":"237d488581a487846e2e2d05241d94daf7efd44a13255b6fbc167e7f6d0ea2a6"}
Sep 12 02:24:43.901173 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:43.901122    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724" podStartSLOduration=1.901103419 podStartE2EDuration="1.901103419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:24:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:24:43.899236615 +0000 UTC m=+2389.426137692" watchObservedRunningTime="2026-09-12 02:24:43.901103419 +0000 UTC m=+2389.428004504"
Sep 12 02:24:52.313845 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.313806    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"]
Sep 12 02:24:52.318167 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.318143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.320630 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.320597    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ecd0309c-kserve-self-signed-certs\""
Sep 12 02:24:52.325480 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.325445    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"]
Sep 12 02:24:52.421493 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.421452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tmp-dir\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.421493 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.421490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tls-certs\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.421703 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.421567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-home\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.421703 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.421623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbk2c\" (UniqueName: \"kubernetes.io/projected/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-kube-api-access-xbk2c\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.421703 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.421639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-dshm\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.421703 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.421660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-model-cache\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.522812 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.522773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-model-cache\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.522981 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.522832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tmp-dir\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.522981 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.522859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tls-certs\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.522981 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.522904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-home\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.522981 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.522962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbk2c\" (UniqueName: \"kubernetes.io/projected/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-kube-api-access-xbk2c\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.523242 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.522986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-dshm\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.523242 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.523182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-model-cache\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.523371 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.523260    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tmp-dir\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.523371 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.523344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-home\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.525193 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.525172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-dshm\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.525546 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.525526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tls-certs\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.530797 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.530779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbk2c\" (UniqueName: \"kubernetes.io/projected/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-kube-api-access-xbk2c\") pod \"e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") " pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.631009 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.630917    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:24:52.911011 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.910978    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"]
Sep 12 02:24:52.912539 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:24:52.912512    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ba41d94_b255_4f21_84d0_6e8b3f2df54d.slice/crio-4fd6f4dfc013c68219f4f1bfc1e3c8a3c5f0e4ecab3ce3ce0b48736176c491f1 WatchSource:0}: Error finding container 4fd6f4dfc013c68219f4f1bfc1e3c8a3c5f0e4ecab3ce3ce0b48736176c491f1: Status 404 returned error can't find the container with id 4fd6f4dfc013c68219f4f1bfc1e3c8a3c5f0e4ecab3ce3ce0b48736176c491f1
Sep 12 02:24:52.941154 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:52.941122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs" event={"ID":"0ba41d94-b255-4f21-84d0-6e8b3f2df54d","Type":"ContainerStarted","Data":"4fd6f4dfc013c68219f4f1bfc1e3c8a3c5f0e4ecab3ce3ce0b48736176c491f1"}
Sep 12 02:24:53.040368 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:53.040332    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:53.040526 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:53.040392    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:53.052587 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:53.052557    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:53.948424 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:53.948386    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs" event={"ID":"0ba41d94-b255-4f21-84d0-6e8b3f2df54d","Type":"ContainerStarted","Data":"df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9"}
Sep 12 02:24:53.960002 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:53.959972    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:24:53.964557 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:53.964512    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs" podStartSLOduration=1.964496842 podStartE2EDuration="1.964496842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:24:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:24:53.96360359 +0000 UTC m=+2399.490504670" watchObservedRunningTime="2026-09-12 02:24:53.964496842 +0000 UTC m=+2399.491397923"
Sep 12 02:24:55.244821 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:55.244786    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:24:55.264782 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:55.264755    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:24:55.900785 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:24:55.900754    2580 scope.go:117] "RemoveContainer" containerID="9c7674b1524b93903e24e6cda516dbba96229be6409c312d40f0eba453877b96"
Sep 12 02:25:02.631646 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:25:02.631611    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:25:02.632029 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:25:02.631664    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:25:02.643782 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:25:02.643754    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:25:03.010117 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:25:03.010080    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:25:55.922693 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:25:55.922658    2580 scope.go:117] "RemoveContainer" containerID="505edce3354dceb73ea134b62615bae8aae4c75a9e2cc7220008d8af122736f9"
Sep 12 02:25:55.931969 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:25:55.931948    2580 scope.go:117] "RemoveContainer" containerID="6a82de88712ad66a556f363dbac2a014fbc7b9f71958d90f6f14b6879218f47b"
Sep 12 02:25:55.941993 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:25:55.941974    2580 scope.go:117] "RemoveContainer" containerID="1d7a61a30fb41ebde35316d97bf66b02457e4153d531c6dad9148e0391c99f92"
Sep 12 02:25:58.812163 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:25:58.812126    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"]
Sep 12 02:26:00.329894 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:00.329862    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"]
Sep 12 02:26:13.832188 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:13.832146    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724" podUID="f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" containerName="main" containerID="cri-o://3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b" gracePeriod=45
Sep 12 02:26:13.948980 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:13.948923    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724" podUID="f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.120:8000/ready\": dial tcp 10.132.0.120:8000: connect: connection refused"
Sep 12 02:26:14.089736 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.089678    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:26:14.155618 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.155582    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-home\") pod \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") "
Sep 12 02:26:14.155789 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.155640    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tmp-dir\") pod \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") "
Sep 12 02:26:14.155789 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.155677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-dshm\") pod \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") "
Sep 12 02:26:14.155789 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.155703    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qww9m\" (UniqueName: \"kubernetes.io/projected/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-kube-api-access-qww9m\") pod \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") "
Sep 12 02:26:14.155789 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.155782    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tls-certs\") pod \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") "
Sep 12 02:26:14.155992 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.155844    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-model-cache\") pod \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\" (UID: \"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7\") "
Sep 12 02:26:14.155992 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.155875    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-home" (OuterVolumeSpecName: "home") pod "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" (UID: "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:26:14.155992 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.155974    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" (UID: "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:26:14.156140 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.156108    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" (UID: "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:26:14.156299 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.156176    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:14.156299 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.156188    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:14.156299 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.156196    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:14.158127 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.158101    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" (UID: "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:26:14.158244 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.158125    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-dshm" (OuterVolumeSpecName: "dshm") pod "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" (UID: "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:26:14.158244 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.158153    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-kube-api-access-qww9m" (OuterVolumeSpecName: "kube-api-access-qww9m") pod "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" (UID: "f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7"). InnerVolumeSpecName "kube-api-access-qww9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:26:14.257600 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.257569    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:14.257600 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.257596    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qww9m\" (UniqueName: \"kubernetes.io/projected/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-kube-api-access-qww9m\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:14.257600 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.257607    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:14.413498 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.413398    2580 generic.go:356] "Generic (PLEG): container finished" podID="f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" containerID="3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b" exitCode=0
Sep 12 02:26:14.413498 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.413453    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"
Sep 12 02:26:14.413498 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.413485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724" event={"ID":"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7","Type":"ContainerDied","Data":"3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b"}
Sep 12 02:26:14.413758 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.413540    2580 scope.go:117] "RemoveContainer" containerID="3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b"
Sep 12 02:26:14.413881 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.413782    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724" event={"ID":"f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7","Type":"ContainerDied","Data":"237d488581a487846e2e2d05241d94daf7efd44a13255b6fbc167e7f6d0ea2a6"}
Sep 12 02:26:14.430192 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.430172    2580 scope.go:117] "RemoveContainer" containerID="3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b"
Sep 12 02:26:14.430525 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:26:14.430508    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b\": container with ID starting with 3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b not found: ID does not exist" containerID="3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b"
Sep 12 02:26:14.430574 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.430532    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b"} err="failed to get container status \"3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b\": rpc error: code = NotFound desc = could not find container \"3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b\": container with ID starting with 3ef9614718614b1a1a9fa059117bb551af2bf16e36a9cda626214219211c905b not found: ID does not exist"
Sep 12 02:26:14.439482 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.439457    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"]
Sep 12 02:26:14.443328 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.443306    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ecd0309c-kserve-bc6bd4bc-g6724"]
Sep 12 02:26:14.979529 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:14.979490    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" path="/var/lib/kubelet/pods/f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7/volumes"
Sep 12 02:26:15.349447 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.349397    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs" podUID="0ba41d94-b255-4f21-84d0-6e8b3f2df54d" containerName="main" containerID="cri-o://df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9" gracePeriod=45
Sep 12 02:26:15.610427 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.610368    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:26:15.669351 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669319    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-dshm\") pod \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") "
Sep 12 02:26:15.669351 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669352    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-home\") pod \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") "
Sep 12 02:26:15.669558 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669398    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tmp-dir\") pod \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") "
Sep 12 02:26:15.669558 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669442    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbk2c\" (UniqueName: \"kubernetes.io/projected/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-kube-api-access-xbk2c\") pod \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") "
Sep 12 02:26:15.669558 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669474    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tls-certs\") pod \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") "
Sep 12 02:26:15.669721 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669584    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-model-cache\") pod \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\" (UID: \"0ba41d94-b255-4f21-84d0-6e8b3f2df54d\") "
Sep 12 02:26:15.669721 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669673    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-home" (OuterVolumeSpecName: "home") pod "0ba41d94-b255-4f21-84d0-6e8b3f2df54d" (UID: "0ba41d94-b255-4f21-84d0-6e8b3f2df54d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:26:15.669828 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669709    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0ba41d94-b255-4f21-84d0-6e8b3f2df54d" (UID: "0ba41d94-b255-4f21-84d0-6e8b3f2df54d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:26:15.669885 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669858    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-model-cache" (OuterVolumeSpecName: "model-cache") pod "0ba41d94-b255-4f21-84d0-6e8b3f2df54d" (UID: "0ba41d94-b255-4f21-84d0-6e8b3f2df54d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:26:15.669949 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669896    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tmp-dir\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:15.669949 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.669912    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-home\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:15.671647 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.671621    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0ba41d94-b255-4f21-84d0-6e8b3f2df54d" (UID: "0ba41d94-b255-4f21-84d0-6e8b3f2df54d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 02:26:15.672006 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.671985    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-dshm" (OuterVolumeSpecName: "dshm") pod "0ba41d94-b255-4f21-84d0-6e8b3f2df54d" (UID: "0ba41d94-b255-4f21-84d0-6e8b3f2df54d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 02:26:15.672006 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.671997    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-kube-api-access-xbk2c" (OuterVolumeSpecName: "kube-api-access-xbk2c") pod "0ba41d94-b255-4f21-84d0-6e8b3f2df54d" (UID: "0ba41d94-b255-4f21-84d0-6e8b3f2df54d"). InnerVolumeSpecName "kube-api-access-xbk2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 02:26:15.770442 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.770409    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-model-cache\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:15.770442 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.770442    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-dshm\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:15.770706 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.770459    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbk2c\" (UniqueName: \"kubernetes.io/projected/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-kube-api-access-xbk2c\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:15.770706 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:15.770473    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ba41d94-b255-4f21-84d0-6e8b3f2df54d-tls-certs\") on node \"ip-10-0-143-251.ec2.internal\" DevicePath \"\""
Sep 12 02:26:16.427343 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.427300    2580 generic.go:356] "Generic (PLEG): container finished" podID="0ba41d94-b255-4f21-84d0-6e8b3f2df54d" containerID="df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9" exitCode=0
Sep 12 02:26:16.427343 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.427344    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"
Sep 12 02:26:16.427853 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.427363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs" event={"ID":"0ba41d94-b255-4f21-84d0-6e8b3f2df54d","Type":"ContainerDied","Data":"df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9"}
Sep 12 02:26:16.427853 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.427404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs" event={"ID":"0ba41d94-b255-4f21-84d0-6e8b3f2df54d","Type":"ContainerDied","Data":"4fd6f4dfc013c68219f4f1bfc1e3c8a3c5f0e4ecab3ce3ce0b48736176c491f1"}
Sep 12 02:26:16.427853 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.427429    2580 scope.go:117] "RemoveContainer" containerID="df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9"
Sep 12 02:26:16.436918 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.436904    2580 scope.go:117] "RemoveContainer" containerID="df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9"
Sep 12 02:26:16.437205 ip-10-0-143-251 kubenswrapper[2580]: E0912 02:26:16.437183    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9\": container with ID starting with df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9 not found: ID does not exist" containerID="df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9"
Sep 12 02:26:16.437299 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.437238    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9"} err="failed to get container status \"df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9\": rpc error: code = NotFound desc = could not find container \"df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9\": container with ID starting with df640d6db76046c21b31140ad0620119178ddfa3d1d2ff8ef2c27a7870c285c9 not found: ID does not exist"
Sep 12 02:26:16.450520 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.450494    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"]
Sep 12 02:26:16.453762 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.453733    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ecd0309c-kserve-6cc4864bc9-frkhs"]
Sep 12 02:26:16.980514 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:16.980477    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ba41d94-b255-4f21-84d0-6e8b3f2df54d" path="/var/lib/kubelet/pods/0ba41d94-b255-4f21-84d0-6e8b3f2df54d/volumes"
Sep 12 02:26:52.317616 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:52.317578    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ffbhv_94a96bcc-5538-465b-a756-eca75abcb493/manager/0.log"
Sep 12 02:26:53.044869 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:53.044835    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578c78994-z59bz_538d7564-cd35-4560-b351-0ad5beb7a992/copy-manifests/0.log"
Sep 12 02:26:53.057334 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:53.057303    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578c78994-z59bz_538d7564-cd35-4560-b351-0ad5beb7a992/manager/0.log"
Sep 12 02:26:53.774180 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:53.774140    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819655-rszcr_b274df06-5749-4259-a56d-2ba1e92aaf1f/cleanup/0.log"
Sep 12 02:26:54.807072 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:54.807039    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-wdhzq_802f816b-fb4a-4c74-ac0b-0b1d998c94b2/manager/0.log"
Sep 12 02:26:54.918719 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:54.918690    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-g58x8_9603f85b-85f9-430c-a4b1-f6087ae8356e/manager/0.log"
Sep 12 02:26:55.035194 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:55.035163    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-nrpjf_f995294f-dfa8-4f3d-b6f1-db595e271c9a/kuadrant-console-plugin/0.log"
Sep 12 02:26:55.162654 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:55.162570    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-6srjq_61cda422-13a9-4648-884d-09f60d8ab85d/manager/0.log"
Sep 12 02:26:55.401072 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:55.401036    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-68wf9_877279e4-3f88-48dd-82ab-6a423910f54d/manager/0.log"
Sep 12 02:26:55.763666 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:55.763619    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn_998fe068-bc7f-4aae-8f7c-d87e468c5db8/istio-proxy/0.log"
Sep 12 02:26:56.602675 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:56.602648    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f959c4b4-5lb49_59059329-6a69-4148-8545-54cb110614d6/router/0.log"
Sep 12 02:26:57.296851 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:26:57.296820    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-qzgxn_f3f4f463-d40d-4ce7-b22c-5fd49621a4f4/main/0.log"
Sep 12 02:27:03.550852 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.550813    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-59ltq/must-gather-ds6wk"]
Sep 12 02:27:03.551317 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.551196    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ba41d94-b255-4f21-84d0-6e8b3f2df54d" containerName="main"
Sep 12 02:27:03.551317 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.551224    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ba41d94-b255-4f21-84d0-6e8b3f2df54d" containerName="main"
Sep 12 02:27:03.551317 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.551248    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" containerName="main"
Sep 12 02:27:03.551317 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.551257    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" containerName="main"
Sep 12 02:27:03.551577 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.551405    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8fe96f3-ac40-46f6-b1fe-3aaa5ef567b7" containerName="main"
Sep 12 02:27:03.551577 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.551427    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ba41d94-b255-4f21-84d0-6e8b3f2df54d" containerName="main"
Sep 12 02:27:03.554522 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.554505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59ltq/must-gather-ds6wk"
Sep 12 02:27:03.556981 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.556960    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59ltq\"/\"openshift-service-ca.crt\""
Sep 12 02:27:03.557914 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.557895    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-59ltq\"/\"default-dockercfg-pg6t6\""
Sep 12 02:27:03.557988 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.557900    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59ltq\"/\"kube-root-ca.crt\""
Sep 12 02:27:03.562255 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.562230    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59ltq/must-gather-ds6wk"]
Sep 12 02:27:03.697080 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.697044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw5v7\" (UniqueName: \"kubernetes.io/projected/4da437ec-6e5a-4eb7-8b16-72b7877270ef-kube-api-access-mw5v7\") pod \"must-gather-ds6wk\" (UID: \"4da437ec-6e5a-4eb7-8b16-72b7877270ef\") " pod="openshift-must-gather-59ltq/must-gather-ds6wk"
Sep 12 02:27:03.697304 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.697101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4da437ec-6e5a-4eb7-8b16-72b7877270ef-must-gather-output\") pod \"must-gather-ds6wk\" (UID: \"4da437ec-6e5a-4eb7-8b16-72b7877270ef\") " pod="openshift-must-gather-59ltq/must-gather-ds6wk"
Sep 12 02:27:03.798573 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.798537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5v7\" (UniqueName: \"kubernetes.io/projected/4da437ec-6e5a-4eb7-8b16-72b7877270ef-kube-api-access-mw5v7\") pod \"must-gather-ds6wk\" (UID: \"4da437ec-6e5a-4eb7-8b16-72b7877270ef\") " pod="openshift-must-gather-59ltq/must-gather-ds6wk"
Sep 12 02:27:03.798755 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.798597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4da437ec-6e5a-4eb7-8b16-72b7877270ef-must-gather-output\") pod \"must-gather-ds6wk\" (UID: \"4da437ec-6e5a-4eb7-8b16-72b7877270ef\") " pod="openshift-must-gather-59ltq/must-gather-ds6wk"
Sep 12 02:27:03.798904 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.798883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4da437ec-6e5a-4eb7-8b16-72b7877270ef-must-gather-output\") pod \"must-gather-ds6wk\" (UID: \"4da437ec-6e5a-4eb7-8b16-72b7877270ef\") " pod="openshift-must-gather-59ltq/must-gather-ds6wk"
Sep 12 02:27:03.806651 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.806581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw5v7\" (UniqueName: \"kubernetes.io/projected/4da437ec-6e5a-4eb7-8b16-72b7877270ef-kube-api-access-mw5v7\") pod \"must-gather-ds6wk\" (UID: \"4da437ec-6e5a-4eb7-8b16-72b7877270ef\") " pod="openshift-must-gather-59ltq/must-gather-ds6wk"
Sep 12 02:27:03.864836 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:03.864800    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59ltq/must-gather-ds6wk"
Sep 12 02:27:04.137744 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:04.137724    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59ltq/must-gather-ds6wk"]
Sep 12 02:27:04.140092 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:27:04.140061    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4da437ec_6e5a_4eb7_8b16_72b7877270ef.slice/crio-9e4745ae274b8ec764d97abad20ec59100e83d27a7d713c4cdd5f89b190846ee WatchSource:0}: Error finding container 9e4745ae274b8ec764d97abad20ec59100e83d27a7d713c4cdd5f89b190846ee: Status 404 returned error can't find the container with id 9e4745ae274b8ec764d97abad20ec59100e83d27a7d713c4cdd5f89b190846ee
Sep 12 02:27:04.142152 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:04.142131    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 02:27:04.703322 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:04.703285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59ltq/must-gather-ds6wk" event={"ID":"4da437ec-6e5a-4eb7-8b16-72b7877270ef","Type":"ContainerStarted","Data":"9e4745ae274b8ec764d97abad20ec59100e83d27a7d713c4cdd5f89b190846ee"}
Sep 12 02:27:05.712011 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:05.711958    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59ltq/must-gather-ds6wk" event={"ID":"4da437ec-6e5a-4eb7-8b16-72b7877270ef","Type":"ContainerStarted","Data":"6005361e384b94bec12ae3e3647b006a977ba8913064953975d902b4d283fd04"}
Sep 12 02:27:05.712505 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:05.712025    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59ltq/must-gather-ds6wk" event={"ID":"4da437ec-6e5a-4eb7-8b16-72b7877270ef","Type":"ContainerStarted","Data":"efa654c1731e71c2ebade8662dc53712b133268aecb41be64116d5389a51d807"}
Sep 12 02:27:05.730870 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:05.730817    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-59ltq/must-gather-ds6wk" podStartSLOduration=2.075581186 podStartE2EDuration="2.730799534s" totalImagesPullingTime="655.218348ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 02:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:27:05.72667546 +0000 UTC m=+2531.253576542" watchObservedRunningTime="2026-09-12 02:27:05.730799534 +0000 UTC m=+2531.257700613"
Sep 12 02:27:06.332625 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:06.332589    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2cbp5_7b590634-a38d-4e33-931d-b7a784f3d07d/global-pull-secret-syncer/0.log"
Sep 12 02:27:06.482528 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:06.482496    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vt5cc_5a1c0a90-2ed6-4a9e-8673-d8022fc9db0c/konnectivity-agent/0.log"
Sep 12 02:27:06.602588 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:06.602505    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-251.ec2.internal_8691b2b790f8ef4d7e8578a8215704ee/haproxy/0.log"
Sep 12 02:27:10.501004 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:10.500973    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-wdhzq_802f816b-fb4a-4c74-ac0b-0b1d998c94b2/manager/0.log"
Sep 12 02:27:10.550867 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:10.550765    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-g58x8_9603f85b-85f9-430c-a4b1-f6087ae8356e/manager/0.log"
Sep 12 02:27:10.591910 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:10.591874    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-nrpjf_f995294f-dfa8-4f3d-b6f1-db595e271c9a/kuadrant-console-plugin/0.log"
Sep 12 02:27:10.637191 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:10.637145    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-6srjq_61cda422-13a9-4648-884d-09f60d8ab85d/manager/0.log"
Sep 12 02:27:10.768252 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:10.768151    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-68wf9_877279e4-3f88-48dd-82ab-6a423910f54d/manager/0.log"
Sep 12 02:27:12.702119 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:12.702004    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-mhx4j_2f22d7ed-e1a6-465d-9ac5-c11f6b1a3d96/cluster-monitoring-operator/0.log"
Sep 12 02:27:12.928155 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:12.928117    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hcpqd_1ee9079f-e83a-4b51-b108-c8f1d3064f52/node-exporter/0.log"
Sep 12 02:27:12.947844 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:12.947813    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hcpqd_1ee9079f-e83a-4b51-b108-c8f1d3064f52/kube-rbac-proxy/0.log"
Sep 12 02:27:12.970129 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:12.970054    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hcpqd_1ee9079f-e83a-4b51-b108-c8f1d3064f52/init-textfile/0.log"
Sep 12 02:27:13.067053 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:13.067021    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-49nx7_832c6f15-f6ad-411e-b33b-43987485def4/kube-rbac-proxy-main/0.log"
Sep 12 02:27:13.085672 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:13.085642    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-49nx7_832c6f15-f6ad-411e-b33b-43987485def4/kube-rbac-proxy-self/0.log"
Sep 12 02:27:13.104549 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:13.104515    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-49nx7_832c6f15-f6ad-411e-b33b-43987485def4/openshift-state-metrics/0.log"
Sep 12 02:27:15.921698 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:15.921656    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/1.log"
Sep 12 02:27:15.932249 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:15.932190    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-m6nx8_a7042ae8-6999-4ac4-beb3-392bb871dc4c/console-operator/2.log"
Sep 12 02:27:16.537355 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:16.537322    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-688bf79bb6-8v5w4_0fec9c0b-98aa-414b-abe5-9483d44f32a4/console/0.log"
Sep 12 02:27:17.075350 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.075308    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"]
Sep 12 02:27:17.081628 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.081592    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.089412 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.089380    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"]
Sep 12 02:27:17.177232 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.177105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-proc\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.177232 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.177178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-podres\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.177232 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.177234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-lib-modules\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.177568 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.177264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-sys\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.177568 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.177325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgcfk\" (UniqueName: \"kubernetes.io/projected/fcb54200-428e-4e31-b632-d9fae959605f-kube-api-access-dgcfk\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.278270 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.278228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-proc\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.278473 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.278306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-podres\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.278473 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.278344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-lib-modules\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.278473 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.278373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-sys\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.278473 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.278404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgcfk\" (UniqueName: \"kubernetes.io/projected/fcb54200-428e-4e31-b632-d9fae959605f-kube-api-access-dgcfk\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.278922 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.278894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-sys\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.279034 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.278892    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-lib-modules\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.279241 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.279190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-podres\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.279436 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.279418    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fcb54200-428e-4e31-b632-d9fae959605f-proc\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.286501 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.286475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgcfk\" (UniqueName: \"kubernetes.io/projected/fcb54200-428e-4e31-b632-d9fae959605f-kube-api-access-dgcfk\") pod \"perf-node-gather-daemonset-xfzc2\" (UID: \"fcb54200-428e-4e31-b632-d9fae959605f\") " pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.300348 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.300316    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-hg7mj_96d615cb-9d90-4ed9-b005-93bbc0f60f94/volume-data-source-validator/0.log"
Sep 12 02:27:17.401568 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.401464    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.701590 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.701559    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"]
Sep 12 02:27:17.704036 ip-10-0-143-251 kubenswrapper[2580]: W0912 02:27:17.703997    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfcb54200_428e_4e31_b632_d9fae959605f.slice/crio-4d6f689978a80d305f430a70ad44439ee5ae073c3fef9e6a271c692a35a61901 WatchSource:0}: Error finding container 4d6f689978a80d305f430a70ad44439ee5ae073c3fef9e6a271c692a35a61901: Status 404 returned error can't find the container with id 4d6f689978a80d305f430a70ad44439ee5ae073c3fef9e6a271c692a35a61901
Sep 12 02:27:17.833180 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.832818    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2" event={"ID":"fcb54200-428e-4e31-b632-d9fae959605f","Type":"ContainerStarted","Data":"b2fb895fd76942685335697a86dd906c0151b8632c721f4213034401a45edcd7"}
Sep 12 02:27:17.833180 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.832881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2" event={"ID":"fcb54200-428e-4e31-b632-d9fae959605f","Type":"ContainerStarted","Data":"4d6f689978a80d305f430a70ad44439ee5ae073c3fef9e6a271c692a35a61901"}
Sep 12 02:27:17.833180 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.833153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:17.850761 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:17.850698    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2" podStartSLOduration=0.850678783 podStartE2EDuration="850.678783ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 02:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 02:27:17.84756592 +0000 UTC m=+2543.374467001" watchObservedRunningTime="2026-09-12 02:27:17.850678783 +0000 UTC m=+2543.377579867"
Sep 12 02:27:18.472417 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:18.472388    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lxb7h_0b83e5ba-8551-45d0-806a-e8c406686503/dns/0.log"
Sep 12 02:27:18.509692 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:18.509660    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lxb7h_0b83e5ba-8551-45d0-806a-e8c406686503/kube-rbac-proxy/0.log"
Sep 12 02:27:18.655104 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:18.655074    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-svxhl_890d8019-49a3-4b97-b6d9-ff60814cb6af/dns-node-resolver/0.log"
Sep 12 02:27:19.256733 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:19.256702    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zjw29_0e050df8-40a2-410e-a291-3177dabbb8f8/node-ca/0.log"
Sep 12 02:27:20.133061 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:20.133026    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn_998fe068-bc7f-4aae-8f7c-d87e468c5db8/istio-proxy/0.log"
Sep 12 02:27:21.124595 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:21.124562    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f959c4b4-5lb49_59059329-6a69-4148-8545-54cb110614d6/router/0.log"
Sep 12 02:27:22.009910 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:22.009862    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cqnvn_5f3f62d7-b8eb-40cb-be5f-f4f4f856e391/serve-healthcheck-canary/0.log"
Sep 12 02:27:22.475033 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:22.474945    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-9fnqn_417f469c-1b81-454a-b5a1-bdd3a402ddbc/insights-operator/0.log"
Sep 12 02:27:22.477526 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:22.477490    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-9fnqn_417f469c-1b81-454a-b5a1-bdd3a402ddbc/insights-operator/1.log"
Sep 12 02:27:22.558050 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:22.558019    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hqh4b_8888f357-81fc-4037-abd4-b9c18d605cc4/kube-rbac-proxy/0.log"
Sep 12 02:27:22.577323 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:22.577294    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hqh4b_8888f357-81fc-4037-abd4-b9c18d605cc4/exporter/0.log"
Sep 12 02:27:22.596147 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:22.596123    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hqh4b_8888f357-81fc-4037-abd4-b9c18d605cc4/extractor/0.log"
Sep 12 02:27:23.852943 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:23.852903    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-59ltq/perf-node-gather-daemonset-xfzc2"
Sep 12 02:27:25.059111 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:25.059079    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ffbhv_94a96bcc-5538-465b-a756-eca75abcb493/manager/0.log"
Sep 12 02:27:25.476632 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:25.476541    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578c78994-z59bz_538d7564-cd35-4560-b351-0ad5beb7a992/manager/0.log"
Sep 12 02:27:25.493467 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:25.493437    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578c78994-z59bz_538d7564-cd35-4560-b351-0ad5beb7a992/copy-manifests/0.log"
Sep 12 02:27:26.952638 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:26.952588    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-b9f9494c8-njtqr_dd05d2c9-02a0-45d3-80df-880ef6059fc3/manager/0.log"
Sep 12 02:27:27.234515 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:27.234404    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn_998fe068-bc7f-4aae-8f7c-d87e468c5db8/istio-proxy/0.log"
Sep 12 02:27:28.181899 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:28.181867    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f959c4b4-5lb49_59059329-6a69-4148-8545-54cb110614d6/router/0.log"
Sep 12 02:27:29.716140 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:29.716108    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cpr4mn_998fe068-bc7f-4aae-8f7c-d87e468c5db8/istio-proxy/0.log"
Sep 12 02:27:30.717630 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:30.717596    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76f959c4b4-5lb49_59059329-6a69-4148-8545-54cb110614d6/router/0.log"
Sep 12 02:27:32.309447 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:32.309396    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-j2m4q_7271188a-6d22-42bb-acd9-f98490da02c8/kube-storage-version-migrator-operator/1.log"
Sep 12 02:27:32.310133 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:32.310114    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-j2m4q_7271188a-6d22-42bb-acd9-f98490da02c8/kube-storage-version-migrator-operator/0.log"
Sep 12 02:27:33.269853 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.269820    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5xxn9_8e4e4495-a8f4-4a7b-9197-d294e0c25280/kube-multus/0.log"
Sep 12 02:27:33.563506 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.563436    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xl2v2_c50a1b1c-0c8c-48b5-bed5-96f6b3591444/kube-multus-additional-cni-plugins/0.log"
Sep 12 02:27:33.590692 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.590656    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xl2v2_c50a1b1c-0c8c-48b5-bed5-96f6b3591444/egress-router-binary-copy/0.log"
Sep 12 02:27:33.638045 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.638020    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xl2v2_c50a1b1c-0c8c-48b5-bed5-96f6b3591444/cni-plugins/0.log"
Sep 12 02:27:33.685066 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.685032    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xl2v2_c50a1b1c-0c8c-48b5-bed5-96f6b3591444/bond-cni-plugin/0.log"
Sep 12 02:27:33.718451 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.718422    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xl2v2_c50a1b1c-0c8c-48b5-bed5-96f6b3591444/routeoverride-cni/0.log"
Sep 12 02:27:33.739793 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.739764    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xl2v2_c50a1b1c-0c8c-48b5-bed5-96f6b3591444/whereabouts-cni-bincopy/0.log"
Sep 12 02:27:33.758562 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.758538    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xl2v2_c50a1b1c-0c8c-48b5-bed5-96f6b3591444/whereabouts-cni/0.log"
Sep 12 02:27:33.918329 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.918255    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w2745_9ae26889-09cb-4e31-9af1-4d560513d4af/network-metrics-daemon/0.log"
Sep 12 02:27:33.932711 ip-10-0-143-251 kubenswrapper[2580]: I0912 02:27:33.932685    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w2745_9ae26889-09cb-4e31-9af1-4d560513d4af/kube-rbac-proxy/0.log"