Sep 16 14:08:00.422007 ip-10-0-137-213 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 14:08:00.768610 ip-10-0-137-213 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:08:00.768610 ip-10-0-137-213 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 14:08:00.768610 ip-10-0-137-213 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:08:00.768610 ip-10-0-137-213 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 14:08:00.768610 ip-10-0-137-213 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:08:00.770236 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.770154    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 14:08:00.772909 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772894    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772911    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772915    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772918    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772923    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772926    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772929    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772933    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772936    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772939    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772944    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772949    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:08:00.772947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772952    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772956    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772959    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772961    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772964    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772967    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772970    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772972    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772975    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772977    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772980    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772983    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772985    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772988    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772990    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772993    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772996    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.772998    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773001    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773003    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:08:00.773231 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773006    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773008    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773011    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773014    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773017    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773019    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773021    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773024    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773026    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773029    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773032    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773035    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773037    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773040    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773043    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773046    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773048    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773051    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773053    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773056    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:08:00.773714 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773059    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773061    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773064    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773066    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773069    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773072    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773074    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773077    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773079    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773082    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773085    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773087    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773089    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773093    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773095    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773098    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773100    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773103    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773105    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773108    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:08:00.774473 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773111    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773114    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773117    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773119    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773122    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773124    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773128    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773131    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773133    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773136    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773138    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773141    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773143    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773146    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773148    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773151    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773153    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773156    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773158    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773161    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:08:00.775045 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773163    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773166    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773168    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773171    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773173    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773176    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.773178    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774485    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774500    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774505    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774510    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774517    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774523    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774528    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774533    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774538    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774543    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774554    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774558    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:08:00.775522 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774582    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774587    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774592    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774599    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774603    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774673    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774797    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774808    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774815    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774821    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774824    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774827    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774829    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774832    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774834    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774837    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774840    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774842    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774845    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:08:00.776023 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774847    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774850    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774852    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774855    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774857    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774860    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774862    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774865    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774868    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774871    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774874    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774877    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774884    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774887    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774889    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774892    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774895    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774897    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774902    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774905    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:08:00.776496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774907    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774910    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774913    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774915    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774918    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774921    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774923    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774926    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774929    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774931    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774934    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774936    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774939    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774942    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774944    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774947    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774949    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774952    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774954    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774957    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:08:00.777010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774960    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774965    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774970    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774974    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774977    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774980    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774983    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774985    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774988    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774991    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774993    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774996    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.774998    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775001    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775004    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775006    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775009    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775011    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775014    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:08:00.777497 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775017    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775019    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775022    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775024    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775027    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775030    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775032    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775035    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775038    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775467    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775474    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775477    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775480    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775483    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775486    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775489    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775492    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775494    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775497    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775500    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:08:00.777982 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775502    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775505    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775507    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775510    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775512    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775515    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775518    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775520    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775522    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775525    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775528    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775531    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775535    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775538    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775542    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775544    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775547    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775550    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775554    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:08:00.778462 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775557    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775560    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775583    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775587    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775592    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775595    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775597    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775600    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775603    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775605    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775608    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775611    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775613    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775617    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775619    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775622    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775625    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775628    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775630    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775639    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:08:00.778954 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775642    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775644    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775647    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775650    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775653    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775655    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775658    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775660    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775663    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775666    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775668    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775670    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775673    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775676    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775678    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775681    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775683    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775686    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775688    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775691    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:08:00.779432 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775693    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775698    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775700    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775703    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775705    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775708    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775710    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775713    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775715    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775718    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775720    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775723    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775732    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775735    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775737    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775740    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775743    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775745    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775748    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775750    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:08:00.779953 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775752    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775755    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775757    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775759    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775763    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775767    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775769    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775772    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.775774    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775839    2566 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775870    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775884    2566 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775889    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775893    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775895    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775899    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775904    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775907    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775910    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775913    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775916    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775919    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 14:08:00.780442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775922    2566 flags.go:64] FLAG: --cgroup-root=""
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775924    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775927    2566 flags.go:64] FLAG: --client-ca-file=""
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775936    2566 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775939    2566 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775950    2566 flags.go:64] FLAG: --cluster-domain=""
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775953    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775956    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775959    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775962    2566 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775966    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775969    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775972    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775975    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775978    2566 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775981    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775984    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775987    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775990    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775994    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.775997    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776000    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776003    2566 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776006    2566 flags.go:64] FLAG: --enable-server="true"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776009    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 14:08:00.780977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776015    2566 flags.go:64] FLAG: --event-burst="100"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776018    2566 flags.go:64] FLAG: --event-qps="50"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776021    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776024    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776027    2566 flags.go:64] FLAG: --eviction-hard=""
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776030    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776033    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776036    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776039    2566 flags.go:64] FLAG: --eviction-soft=""
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776042    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776045    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776054    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776059    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776062    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776064    2566 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776067    2566 flags.go:64] FLAG: --feature-gates=""
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776071    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776074    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776076    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776080    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776082    2566 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776085    2566 flags.go:64] FLAG: --help="false"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776088    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-137-213.ec2.internal"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776091    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 14:08:00.781632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776094    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776097    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776100    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776103    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776107    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776110    2566 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776113    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776116    2566 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776119    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776122    2566 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776125    2566 flags.go:64] FLAG: --kube-reserved=""
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776128    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776131    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776134    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776136    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776139    2566 flags.go:64] FLAG: --lock-file=""
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776142    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776145    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776148    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776157    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776168    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776171    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776173    2566 flags.go:64] FLAG: --logging-format="text"
Sep 16 14:08:00.782312 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776176    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776179    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776182    2566 flags.go:64] FLAG: --manifest-url=""
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776185    2566 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776189    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776192    2566 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776196    2566 flags.go:64] FLAG: --max-pods="110"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776198    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776201    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776204    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776207    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776210    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776212    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776217    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776223    2566 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776226    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776229    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776232    2566 flags.go:64] FLAG: --pod-cidr=""
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776235    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776240    2566 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776243    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776246    2566 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776248    2566 flags.go:64] FLAG: --port="10250"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776251    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 14:08:00.782934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776254    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ba724afc645bacee"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776257    2566 flags.go:64] FLAG: --qos-reserved=""
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776260    2566 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776262    2566 flags.go:64] FLAG: --register-node="true"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776265    2566 flags.go:64] FLAG: --register-with-taints=""
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776269    2566 flags.go:64] FLAG: --registry-burst="10"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776279    2566 flags.go:64] FLAG: --registry-qps="5"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776281    2566 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776284    2566 flags.go:64] FLAG: --reserved-memory=""
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776288    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776291    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776294    2566 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776297    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776299    2566 flags.go:64] FLAG: --runonce="false"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776302    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776306    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776309    2566 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776312    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776314    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776317    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776320    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776324    2566 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776327    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776330    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776333    2566 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776336    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 14:08:00.783511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776339    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776341    2566 flags.go:64] FLAG: --system-cgroups=""
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776344    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776349    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776352    2566 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776354    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776361    2566 flags.go:64] FLAG: --tls-min-version=""
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776364    2566 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776367    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776369    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776372    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776375    2566 flags.go:64] FLAG: --v="2"
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776380    2566 flags.go:64] FLAG: --version="false"
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776386    2566 flags.go:64] FLAG: --vmodule=""
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776389    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.776393    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776510    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776514    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776518    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776520    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776523    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776526    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776528    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:08:00.784164 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776531    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776534    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776537    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776539    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776546    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776556    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776559    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776576    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776581    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776584    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776587    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776590    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776592    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776595    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776598    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776600    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776603    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776605    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776608    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:08:00.784759 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776610    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776613    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776617    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776620    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776623    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776626    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776629    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776631    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776633    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776636    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776639    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776641    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776644    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776646    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776649    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776651    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776655    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776661    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776664    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:08:00.785241 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776666    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776669    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776672    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776675    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776677    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776680    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776682    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776685    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776687    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776690    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776692    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776694    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776697    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776699    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776702    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776705    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776709    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776713    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776716    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:08:00.785730 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776719    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776722    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776725    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776728    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776730    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776733    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776735    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776738    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776740    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776742    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776745    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776749    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776751    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776754    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776756    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776759    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776761    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776763    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776766    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776768    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:08:00.786205 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776771    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776773    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776775    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776778    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776781    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776783    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776786    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776788    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776792    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776795    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776797    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776801    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776805    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776808    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.776811    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:08:00.786709 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.777259    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.784311    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.784327    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.784948    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.784957    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785014    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785018    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785022    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785025    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785029    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785032    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785035    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785038    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785042    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785045    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785047    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785050    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785053    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785056    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:08:00.787081 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785058    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785061    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785063    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785066    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785070    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785075    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785078    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785081    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785084    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785088    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785092    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785095    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785098    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785101    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785104    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785108    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785111    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785114    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785117    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:08:00.787548 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785119    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785122    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785124    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785127    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785131    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785134    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785138    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785140    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785143    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785146    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785149    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785151    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785154    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785157    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785160    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785162    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785165    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785168    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785170    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:08:00.788110 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785173    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785175    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785178    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785180    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785183    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785185    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785188    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785190    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785193    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785196    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785199    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785202    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785205    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785207    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785210    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785212    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785215    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785217    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785219    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785222    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:08:00.788635 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785224    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785227    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785229    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785232    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785234    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785237    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785239    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785242    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785245    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785247    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785250    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785253    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785255    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785258    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785261    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785264    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785266    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785269    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785271    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785274    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:08:00.789113 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785276    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785279    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785281    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785284    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785287    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785289    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785292    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.785297    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785416    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785421    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785424    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785427    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785430    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785433    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785435    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:08:00.789620 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785438    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785440    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785443    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785445    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785448    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785450    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785453    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785455    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785459    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785463    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785466    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785469    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785471    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785474    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785477    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785479    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785482    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785485    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785487    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785490    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:08:00.790010 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785493    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785496    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785499    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785501    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785504    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785506    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785509    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785511    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785514    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785516    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785518    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785521    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785523    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785526    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785528    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785531    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785533    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785536    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785538    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785540    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:08:00.790491 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785543    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785545    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785548    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785551    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785553    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785556    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785559    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785561    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785583    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785586    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785588    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785591    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785593    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785596    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785599    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785602    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785604    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785606    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785609    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785611    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:08:00.790986 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785614    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785616    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785619    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785621    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785624    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785626    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785630    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785633    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785636    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785638    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785641    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785643    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785646    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785648    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785651    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785654    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785656    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785659    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785662    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785664    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:08:00.791496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785666    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785669    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785671    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785675    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785679    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785682    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785684    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785688    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785691    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785694    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785696    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:00.785699    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.785704    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:08:00.792078 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.785853    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 14:08:00.792405 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.788374    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 14:08:00.792405 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.789352    2566 server.go:1019] "Starting client certificate rotation"
Sep 16 14:08:00.792405 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.789446    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 14:08:00.792405 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.789478    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 14:08:00.792405 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.791548    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 14:08:00.793146 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.793127    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 14:08:00.806871 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.806847    2566 log.go:25] "Validated CRI v1 runtime API"
Sep 16 14:08:00.810164 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.810148    2566 log.go:25] "Validated CRI v1 image API"
Sep 16 14:08:00.811091 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.811066    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 14:08:00.816043 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.816022    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 14:08:00.817128 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.817106    2566 fs.go:135] Filesystem UUIDs: map[329cfbdc-9602-4f8d-bca8-828882f6fae6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ca5f72c4-f128-411e-b735-6b4bb8f90cd0:/dev/nvme0n1p3]
Sep 16 14:08:00.817200 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.817128    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 14:08:00.823726 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.823600    2566 manager.go:216] Machine: {Timestamp:2026-09-16 14:08:00.821863891 +0000 UTC m=+0.299095977 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100664 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24130fba382c9586136d044d4e9da5 SystemUUID:ec24130f-ba38-2c95-8613-6d044d4e9da5 BootID:56f78f4e-1fa4-4f28-822e-338c8e557299 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fd:a9:9d:a1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fd:a9:9d:a1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:bd:69:a2:a5:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 14:08:00.823726 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.823716    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 14:08:00.823893 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.823878    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 14:08:00.827527 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.827504    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 14:08:00.827680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.827530    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 14:08:00.828376 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.828366    2566 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 14:08:00.828407 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.828379    2566 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 14:08:00.828407 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.828387    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 14:08:00.829209 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.829199    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 14:08:00.830437 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.830426    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 14:08:00.834895 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.834882    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 14:08:00.835012 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.835002    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 14:08:00.835055 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.835033    2566 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 14:08:00.835055 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.835051    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 14:08:00.835125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.835062    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 14:08:00.835125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.835074    2566 kubelet.go:398] "Adding apiserver pod source"
Sep 16 14:08:00.835125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.835084    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 14:08:00.835436 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.835424    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 14:08:00.835488 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.835442    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 14:08:00.837832 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.837810    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 14:08:00.839901 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.839885    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 14:08:00.839971 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.839916    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 14:08:00.841909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841892    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 14:08:00.841959 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841918    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 14:08:00.841959 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841925    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 14:08:00.841959 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841930    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 14:08:00.841959 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841939    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 14:08:00.841959 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841944    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 14:08:00.841959 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841958    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 14:08:00.842175 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841967    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 14:08:00.842175 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841977    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 14:08:00.842175 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841982    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 14:08:00.842175 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841991    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 14:08:00.842175 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.841996    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 14:08:00.842903 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.842892    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 14:08:00.842903 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.842902    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 14:08:00.843999 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.843974    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 14:08:00.844149 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.844128    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 14:08:00.846267 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.846253    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 14:08:00.846920 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.846908    2566 server.go:1268] "Started kubelet"
Sep 16 14:08:00.850553 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.850520    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 14:08:00.851160 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.850820    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 14:08:00.851160 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.850902    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 14:08:00.851675 ip-10-0-137-213 systemd[1]: Started Kubernetes Kubelet.
Sep 16 14:08:00.853131 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.853115    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 14:08:00.854748 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.854732    2566 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 14:08:00.857023 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.856998    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 14:08:00.857817 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.856709    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-213.ec2.internal.18d5d252fe65e1c6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-213.ec2.internal,UID:ip-10-0-137-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-213.ec2.internal,},FirstTimestamp:2026-09-16 14:08:00.846881222 +0000 UTC m=+0.324113289,LastTimestamp:2026-09-16 14:08:00.846881222 +0000 UTC m=+0.324113289,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-213.ec2.internal,}"
Sep 16 14:08:00.858203 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.858183    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2dw64"
Sep 16 14:08:00.859443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.859417    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 14:08:00.860096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.860079    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 14:08:00.860181 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.860168    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:00.860857 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.860843    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 14:08:00.860857 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.860860    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 14:08:00.860986 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.860843    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 14:08:00.860986 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.860943    2566 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 14:08:00.860986 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.860955    2566 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 14:08:00.861103 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.861093    2566 factory.go:153] Registering CRI-O factory
Sep 16 14:08:00.861147 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.861133    2566 factory.go:223] Registration of the crio container factory successfully
Sep 16 14:08:00.861200 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.861187    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 14:08:00.861200 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.861197    2566 factory.go:55] Registering systemd factory
Sep 16 14:08:00.861295 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.861206    2566 factory.go:223] Registration of the systemd container factory successfully
Sep 16 14:08:00.861295 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.861230    2566 factory.go:103] Registering Raw factory
Sep 16 14:08:00.861295 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.861244    2566 manager.go:1196] Started watching for new ooms in manager
Sep 16 14:08:00.861605 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.861592    2566 manager.go:349] Starting recovery of all containers
Sep 16 14:08:00.863951 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.863923    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 14:08:00.864099 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.864078    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 14:08:00.870082 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.870066    2566 manager.go:354] Recovery completed
Sep 16 14:08:00.870273 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.870253    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2dw64"
Sep 16 14:08:00.875919 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.875904    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:08:00.877451 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.877434    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:08:00.877521 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.877467    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:08:00.877521 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.877482    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:08:00.878077 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.878064    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 14:08:00.878077 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.878076    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 14:08:00.878191 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.878091    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 14:08:00.878191 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.878077    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:08:00.878836 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.878822    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:08:00.878910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.878850    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:08:00.878910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.878863    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:08:00.879785 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.879720    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-213.ec2.internal.18d5d253003858a7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-213.ec2.internal,UID:ip-10-0-137-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-213.ec2.internal,},FirstTimestamp:2026-09-16 14:08:00.877451431 +0000 UTC m=+0.354683498,LastTimestamp:2026-09-16 14:08:00.877451431 +0000 UTC m=+0.354683498,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-213.ec2.internal,}"
Sep 16 14:08:00.880119 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.880108    2566 policy_none.go:49] "None policy: Start"
Sep 16 14:08:00.880159 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.880123    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 14:08:00.880159 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.880133    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 14:08:00.882087 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.882076    2566 policy_none.go:47] "Start"
Sep 16 14:08:00.917162 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.917146    2566 manager.go:337] "Starting Device Plugin manager"
Sep 16 14:08:00.917295 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.917194    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 14:08:00.917295 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.917208    2566 server.go:87] "Starting device plugin registration server"
Sep 16 14:08:00.917512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.917499    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 14:08:00.917740 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.917517    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 14:08:00.917740 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.917701    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 14:08:00.917877 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.917790    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 14:08:00.917877 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.917800    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 14:08:00.918272 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.918251    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 14:08:00.918368 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.918324    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:00.976032 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.975985    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 14:08:00.977442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.977421    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 14:08:00.977554 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.977474    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 14:08:00.977554 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.977503    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 14:08:00.977659 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:00.977553    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 14:08:00.980792 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:00.980773    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:01.017856 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.017826    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:08:01.018909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.018838    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:08:01.018909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.018868    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:08:01.018909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.018882    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:08:01.018909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.018911    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.031453 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.031433    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.031517 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.031459    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-213.ec2.internal\": node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.078611 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.078557    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal"]
Sep 16 14:08:01.078756 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.078655    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:08:01.080131 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.080108    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:08:01.080242 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.080141    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:08:01.080242 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.080158    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:08:01.080505 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.080486    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.082343 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.082330    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:08:01.082476 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.082464    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.082513 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.082491    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:08:01.083053 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.083036    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:08:01.083148 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.083061    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:08:01.083148 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.083075    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:08:01.083148 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.083129    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:08:01.083234 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.083155    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:08:01.083234 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.083169    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:08:01.085345 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.085328    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.085415 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.085360    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:08:01.086098 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.086080    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:08:01.086176 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.086116    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:08:01.086176 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.086132    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:08:01.102161 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.102139    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-213.ec2.internal\" not found" node="ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.105654 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.105636    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-213.ec2.internal\" not found" node="ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.163024 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.162997    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f4943e5a79d3f51d24a525e5e47d75f0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal\" (UID: \"f4943e5a79d3f51d24a525e5e47d75f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.163024 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.163029    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4943e5a79d3f51d24a525e5e47d75f0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal\" (UID: \"f4943e5a79d3f51d24a525e5e47d75f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.163208 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.163047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf01562adb7ffda058010a76980228c7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-213.ec2.internal\" (UID: \"bf01562adb7ffda058010a76980228c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.180612 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.180585    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.263801 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.263776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f4943e5a79d3f51d24a525e5e47d75f0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal\" (UID: \"f4943e5a79d3f51d24a525e5e47d75f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.263872 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.263811    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4943e5a79d3f51d24a525e5e47d75f0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal\" (UID: \"f4943e5a79d3f51d24a525e5e47d75f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.263872 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.263835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf01562adb7ffda058010a76980228c7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-213.ec2.internal\" (UID: \"bf01562adb7ffda058010a76980228c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.263936 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.263895    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf01562adb7ffda058010a76980228c7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-213.ec2.internal\" (UID: \"bf01562adb7ffda058010a76980228c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.263936 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.263913    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f4943e5a79d3f51d24a525e5e47d75f0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal\" (UID: \"f4943e5a79d3f51d24a525e5e47d75f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.263936 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.263897    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4943e5a79d3f51d24a525e5e47d75f0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal\" (UID: \"f4943e5a79d3f51d24a525e5e47d75f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.281391 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.281329    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.381986 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.381947    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.404133 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.404104    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.409134 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.408816    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:01.482712 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.482681    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.583309 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.583239    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.683817 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.683790    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.757212 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.757182    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:01.770491 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.770472    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:01.784233 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.784202    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.789376 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.789358    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 14:08:01.789501 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.789485    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 14:08:01.789533 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.789518    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 14:08:01.789583 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.789523    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 14:08:01.878128 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.878057    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 14:03:01 +0000 UTC" deadline="2028-03-22 21:28:00.118714756 +0000 UTC"
Sep 16 14:08:01.878128 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:01.878086    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13279h19m58.240630938s"
Sep 16 14:08:01.885241 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.885206    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:01.985847 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:01.985807    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:02.086326 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:02.086291    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:02.187037 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:02.186811    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:02.254525 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:02.254491    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4943e5a79d3f51d24a525e5e47d75f0.slice/crio-4d7b0146fb647e335153105f7d624d5f4e30d8ba88b77cfeb9f364d04f038828 WatchSource:0}: Error finding container 4d7b0146fb647e335153105f7d624d5f4e30d8ba88b77cfeb9f364d04f038828: Status 404 returned error can't find the container with id 4d7b0146fb647e335153105f7d624d5f4e30d8ba88b77cfeb9f364d04f038828
Sep 16 14:08:02.261538 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.261520    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:08:02.287939 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:02.287903    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:02.388778 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:02.388753    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-213.ec2.internal\" not found"
Sep 16 14:08:02.433845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.433819    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:02.460752 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.460692    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:02.470637 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.470616    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 14:08:02.471394 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.471381    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"
Sep 16 14:08:02.482045 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.482027    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 14:08:02.665705 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:02.665677    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf01562adb7ffda058010a76980228c7.slice/crio-5d5afa9ea7d79ba053cdcdefa9b5df3ca1b7e8580e7e12a1023c9fb513f430de WatchSource:0}: Error finding container 5d5afa9ea7d79ba053cdcdefa9b5df3ca1b7e8580e7e12a1023c9fb513f430de: Status 404 returned error can't find the container with id 5d5afa9ea7d79ba053cdcdefa9b5df3ca1b7e8580e7e12a1023c9fb513f430de
Sep 16 14:08:02.836511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.836483    2566 apiserver.go:52] "Watching apiserver"
Sep 16 14:08:02.842511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.842487    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 14:08:02.842903 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.842876    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-7lcv6","openshift-network-diagnostics/network-check-target-8v4g2","openshift-network-operator/iptables-alerter-4dm64","openshift-ovn-kubernetes/ovnkube-node-mb584","kube-system/global-pull-secret-syncer-97f7n","kube-system/konnectivity-agent-r6qxk","kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm","openshift-cluster-node-tuning-operator/tuned-rvthp","openshift-multus/multus-z8r9s","openshift-multus/network-metrics-daemon-sjjm9","openshift-dns/node-resolver-ptf79","openshift-image-registry/node-ca-kw5db","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal"]
Sep 16 14:08:02.845642 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.845616    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.847991 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.847965    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.848095 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.848058    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 14:08:02.848319 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.848301    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 14:08:02.848382 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.848366    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8zrgz\""
Sep 16 14:08:02.848425 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.848301    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 14:08:02.850063 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.850001    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p4grs\""
Sep 16 14:08:02.850172 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.850121    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 14:08:02.850172 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.850122    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 14:08:02.850484 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.850457    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.852497 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.852476    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 14:08:02.852607 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.852549    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vb8p8\""
Sep 16 14:08:02.852607 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.852558    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 14:08:02.852752 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.852549    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 14:08:02.853852 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.853008    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.853852 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.853487    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 14:08:02.855490 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.855469    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:02.855587 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.855544    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 14:08:02.855679 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.855564    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 14:08:02.855679 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.855567    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 14:08:02.855786 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.855713    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wqqt2\""
Sep 16 14:08:02.855786 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:02.855539    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:02.855978 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.855954    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 14:08:02.856202 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.856098    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 14:08:02.856202 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.856126    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 14:08:02.858169 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.858146    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:02.861081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.860180    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 14:08:02.861081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.860451    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 14:08:02.861081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.860470    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 14:08:02.861081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.860712    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c9z8x\""
Sep 16 14:08:02.861081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.861020    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.863398 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.863382    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.864680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.864610    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 14:08:02.864909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.864892    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 14:08:02.865027 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.865007    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vcqfc\""
Sep 16 14:08:02.867464 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.865363    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:08:02.867464 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.866395    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 14:08:02.867464 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.866523    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9hfbg\""
Sep 16 14:08:02.867464 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.866596    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:08:02.867464 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.867002    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:02.870554 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.870532    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:02.870668 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:02.870644    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:02.871428 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.871356    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 14:08:02.871428 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.871362    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 14:08:02.871580 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.871533    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rqqx6\""
Sep 16 14:08:02.873321 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.873295    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:02.873403 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:02.873373    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:02.873829 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.873805    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-var-lib-openvswitch\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.873921 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.873845    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcdnf\" (UniqueName: \"kubernetes.io/projected/0d67f7d8-3265-4271-8e17-9cf0b1a6ea73-kube-api-access-jcdnf\") pod \"iptables-alerter-4dm64\" (UID: \"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73\") " pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.873921 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.873872    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-device-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.873921 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.873910    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-sys-fs\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.874094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.873936    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-cnibin\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.874094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.873960    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-run-netns\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.874094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.873992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-tuned\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874015    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-run-netns\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.874094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-run-systemd\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.874094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874064    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cfb2ccd-544b-4d36-8765-e1530714db1d-ovnkube-script-lib\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.874094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874087    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874111    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-hostroot\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874142    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-run-multus-certs\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874215    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cc6e861-6a80-4af5-98fa-8e19d088e530-konnectivity-ca\") pod \"konnectivity-agent-r6qxk\" (UID: \"9cc6e861-6a80-4af5-98fa-8e19d088e530\") " pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874256    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-sysconfig\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874287    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1593320-c5df-4f1b-bb35-b90738b70602-dbus\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874312    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-run-k8s-cni-cncf-io\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874335    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-modprobe-d\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874350    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-sys\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874365    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-var-lib-kubelet\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874387    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/978fd76a-8903-4505-b2f5-d8b070cbcd9a-tmp\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874410    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1593320-c5df-4f1b-bb35-b90738b70602-kubelet-config\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874463    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-cni-dir\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874488    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7bql\" (UniqueName: \"kubernetes.io/projected/c611f22c-2026-4fb5-b56b-aead114a76a0-kube-api-access-v7bql\") pod \"node-resolver-ptf79\" (UID: \"c611f22c-2026-4fb5-b56b-aead114a76a0\") " pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874505    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-systemd-units\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874565    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d67f7d8-3265-4271-8e17-9cf0b1a6ea73-host-slash\") pod \"iptables-alerter-4dm64\" (UID: \"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73\") " pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874619    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-run\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874650    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-lib-modules\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874702    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c611f22c-2026-4fb5-b56b-aead114a76a0-tmp-dir\") pod \"node-resolver-ptf79\" (UID: \"c611f22c-2026-4fb5-b56b-aead114a76a0\") " pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874725    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-var-lib-kubelet\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cc6e861-6a80-4af5-98fa-8e19d088e530-agent-certs\") pod \"konnectivity-agent-r6qxk\" (UID: \"9cc6e861-6a80-4af5-98fa-8e19d088e530\") " pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874779    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-sysctl-d\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874803    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-systemd\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874838    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c611f22c-2026-4fb5-b56b-aead114a76a0-hosts-file\") pod \"node-resolver-ptf79\" (UID: \"c611f22c-2026-4fb5-b56b-aead114a76a0\") " pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874861    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-cni-bin\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874884    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-var-lib-cni-multus\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.874910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874908    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-cni-netd\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874930    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cfb2ccd-544b-4d36-8765-e1530714db1d-ovn-node-metrics-cert\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874954    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-socket-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.874978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-registration-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-cni-binary-copy\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875024    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-run-ovn-kubernetes\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875048    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-sysctl-conf\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-etc-selinux\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875088    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq4h9\" (UniqueName: \"kubernetes.io/projected/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-kube-api-access-qq4h9\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875104    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-run-ovn\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875121    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cfb2ccd-544b-4d36-8765-e1530714db1d-ovnkube-config\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875136    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbvw6\" (UniqueName: \"kubernetes.io/projected/978fd76a-8903-4505-b2f5-d8b070cbcd9a-kube-api-access-tbvw6\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875161    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmg4k\" (UniqueName: \"kubernetes.io/projected/5cfb2ccd-544b-4d36-8765-e1530714db1d-kube-api-access-nmg4k\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875178    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875193    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d67f7d8-3265-4271-8e17-9cf0b1a6ea73-iptables-alerter-script\") pod \"iptables-alerter-4dm64\" (UID: \"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73\") " pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875209    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7ntp\" (UniqueName: \"kubernetes.io/projected/f5236695-aa13-43d5-925b-e40362385900-kube-api-access-c7ntp\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.875529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875223    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-var-lib-cni-bin\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-host\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875274    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-slash\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875306    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-node-log\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875333    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cfb2ccd-544b-4d36-8765-e1530714db1d-env-overrides\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875359    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-run-openvswitch\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875384    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-log-socket\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875411    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-conf-dir\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875460    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-daemon-config\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-etc-kubernetes\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875539    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-kubernetes\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875563    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-kubelet\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875611    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-etc-openvswitch\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875636    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-system-cni-dir\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875663    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-socket-dir-parent\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.876154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.875715    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:02.878092 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.877807    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 14:08:02.878092 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.877921    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 14:08:02.878092 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.878036    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gnrpc\""
Sep 16 14:08:02.878092 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.878070    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 14:08:02.878699 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.878680    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 14:08:02.909775 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.909754    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wztzj"
Sep 16 14:08:02.919819 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.919799    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wztzj"
Sep 16 14:08:02.962370 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.962340    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 14:08:02.976037 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.975970    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5szv\" (UniqueName: \"kubernetes.io/projected/5b1d453d-626a-4d08-ad51-474bfdce1e13-kube-api-access-g5szv\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:02.976037 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976012    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-registration-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.976199 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976038    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-cni-binary-copy\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.976199 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976139    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-registration-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.976199 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976154    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b1d453d-626a-4d08-ad51-474bfdce1e13-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:02.976347 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976223    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-run-ovn-kubernetes\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.976347 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976274    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-run-ovn-kubernetes\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.976347 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976296    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-sysctl-conf\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.976347 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976329    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:02.976542 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976356    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-etc-selinux\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.976542 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976384    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq4h9\" (UniqueName: \"kubernetes.io/projected/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-kube-api-access-qq4h9\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.976542 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-run-ovn\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.976542 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976440    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cfb2ccd-544b-4d36-8765-e1530714db1d-ovnkube-config\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.976542 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976498    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-run-ovn\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.976542 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976508    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-sysctl-conf\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.976542 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976535    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbvw6\" (UniqueName: \"kubernetes.io/projected/978fd76a-8903-4505-b2f5-d8b070cbcd9a-kube-api-access-tbvw6\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976544    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-etc-selinux\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976564    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmg4k\" (UniqueName: \"kubernetes.io/projected/5cfb2ccd-544b-4d36-8765-e1530714db1d-kube-api-access-nmg4k\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976662    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d67f7d8-3265-4271-8e17-9cf0b1a6ea73-iptables-alerter-script\") pod \"iptables-alerter-4dm64\" (UID: \"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73\") " pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976717    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7ntp\" (UniqueName: \"kubernetes.io/projected/f5236695-aa13-43d5-925b-e40362385900-kube-api-access-c7ntp\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976747    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-var-lib-cni-bin\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-host\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976817    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-cni-binary-copy\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976889    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-var-lib-cni-bin\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976805    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:02.976918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976909    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-host\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976953    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-slash\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.976982    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-node-log\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977010    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cfb2ccd-544b-4d36-8765-e1530714db1d-env-overrides\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-run-openvswitch\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977051    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-node-log\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-log-socket\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:02.977093    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977119    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-log-socket\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977132    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-run-openvswitch\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977164    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977194    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-conf-dir\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977293    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-daemon-config\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977301    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-etc-kubernetes\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:02.977397    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret podName:c1593320-c5df-4f1b-bb35-b90738b70602 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:03.477325038 +0000 UTC m=+2.954557116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret") pod "global-pull-secret-syncer-97f7n" (UID: "c1593320-c5df-4f1b-bb35-b90738b70602") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977379    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-conf-dir\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.977465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977436    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-etc-kubernetes\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-kubernetes\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977556    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cfb2ccd-544b-4d36-8765-e1530714db1d-env-overrides\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977523    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b1d453d-626a-4d08-ad51-474bfdce1e13-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977599    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-slash\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977630    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-kubelet\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977622    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d67f7d8-3265-4271-8e17-9cf0b1a6ea73-iptables-alerter-script\") pod \"iptables-alerter-4dm64\" (UID: \"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73\") " pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977668    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-etc-openvswitch\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977701    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-etc-openvswitch\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-system-cni-dir\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977737    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-kubernetes\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977228    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cfb2ccd-544b-4d36-8765-e1530714db1d-ovnkube-config\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977792    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-socket-dir-parent\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977819    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-kubelet\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977854    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b1d453d-626a-4d08-ad51-474bfdce1e13-system-cni-dir\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-system-cni-dir\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977923    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/087e2244-c0c9-4141-8f5f-b94636c32bf4-serviceca\") pod \"node-ca-kw5db\" (UID: \"087e2244-c0c9-4141-8f5f-b94636c32bf4\") " pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:02.978235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977946    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-var-lib-openvswitch\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977946    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-socket-dir-parent\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977975    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcdnf\" (UniqueName: \"kubernetes.io/projected/0d67f7d8-3265-4271-8e17-9cf0b1a6ea73-kube-api-access-jcdnf\") pod \"iptables-alerter-4dm64\" (UID: \"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73\") " pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.977999    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-var-lib-openvswitch\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-device-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978063    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-daemon-config\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978070    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-sys-fs\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978105    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-cnibin\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978129    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-sys-fs\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978150    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-device-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-run-netns\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978218    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-cnibin\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978219    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-tuned\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978258    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/087e2244-c0c9-4141-8f5f-b94636c32bf4-host\") pod \"node-ca-kw5db\" (UID: \"087e2244-c0c9-4141-8f5f-b94636c32bf4\") " pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978285    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-run-netns\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-run-netns\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-run-systemd\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978393    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-run-netns\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.979083 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cfb2ccd-544b-4d36-8765-e1530714db1d-ovnkube-script-lib\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978452    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-run-systemd\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978472    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978501    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-hostroot\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978527    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-run-multus-certs\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978559    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cc6e861-6a80-4af5-98fa-8e19d088e530-konnectivity-ca\") pod \"konnectivity-agent-r6qxk\" (UID: \"9cc6e861-6a80-4af5-98fa-8e19d088e530\") " pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978620    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-hostroot\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978587    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-sysconfig\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1593320-c5df-4f1b-bb35-b90738b70602-dbus\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978692    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-run-multus-certs\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978761    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-run-k8s-cni-cncf-io\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978768    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-sysconfig\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978792    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-modprobe-d\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978778    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-sys\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978915    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1593320-c5df-4f1b-bb35-b90738b70602-dbus\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978657    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.978981    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-run-k8s-cni-cncf-io\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.979953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979047    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-modprobe-d\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979075    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cfb2ccd-544b-4d36-8765-e1530714db1d-ovnkube-script-lib\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979156    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-sys\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-var-lib-kubelet\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/978fd76a-8903-4505-b2f5-d8b070cbcd9a-tmp\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979236    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1593320-c5df-4f1b-bb35-b90738b70602-kubelet-config\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979262    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-cni-dir\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979292    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp7j5\" (UniqueName: \"kubernetes.io/projected/087e2244-c0c9-4141-8f5f-b94636c32bf4-kube-api-access-wp7j5\") pod \"node-ca-kw5db\" (UID: \"087e2244-c0c9-4141-8f5f-b94636c32bf4\") " pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979317    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1593320-c5df-4f1b-bb35-b90738b70602-kubelet-config\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979210    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-var-lib-kubelet\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7bql\" (UniqueName: \"kubernetes.io/projected/c611f22c-2026-4fb5-b56b-aead114a76a0-kube-api-access-v7bql\") pod \"node-resolver-ptf79\" (UID: \"c611f22c-2026-4fb5-b56b-aead114a76a0\") " pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979379    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-systemd-units\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d67f7d8-3265-4271-8e17-9cf0b1a6ea73-host-slash\") pod \"iptables-alerter-4dm64\" (UID: \"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73\") " pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979442    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-run\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979406    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cc6e861-6a80-4af5-98fa-8e19d088e530-konnectivity-ca\") pod \"konnectivity-agent-r6qxk\" (UID: \"9cc6e861-6a80-4af5-98fa-8e19d088e530\") " pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979468    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-lib-modules\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979525    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-systemd-units\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.980845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979570    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnz6m\" (UniqueName: \"kubernetes.io/projected/e3059f22-c414-4a85-9a55-47485f8a88b3-kube-api-access-dnz6m\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979612    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c611f22c-2026-4fb5-b56b-aead114a76a0-tmp-dir\") pod \"node-resolver-ptf79\" (UID: \"c611f22c-2026-4fb5-b56b-aead114a76a0\") " pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979628    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-run\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-var-lib-kubelet\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979663    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-multus-cni-dir\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cc6e861-6a80-4af5-98fa-8e19d088e530-agent-certs\") pod \"konnectivity-agent-r6qxk\" (UID: \"9cc6e861-6a80-4af5-98fa-8e19d088e530\") " pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979703    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-sysctl-d\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-var-lib-kubelet\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-systemd\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979733    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-lib-modules\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979795    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b1d453d-626a-4d08-ad51-474bfdce1e13-cnibin\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979799    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d67f7d8-3265-4271-8e17-9cf0b1a6ea73-host-slash\") pod \"iptables-alerter-4dm64\" (UID: \"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73\") " pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c611f22c-2026-4fb5-b56b-aead114a76a0-hosts-file\") pod \"node-resolver-ptf79\" (UID: \"c611f22c-2026-4fb5-b56b-aead114a76a0\") " pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-systemd\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979939    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-cni-bin\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979980    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-sysctl-d\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.979985    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-var-lib-cni-multus\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980050    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b1d453d-626a-4d08-ad51-474bfdce1e13-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:02.981680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980057    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-host-var-lib-cni-multus\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.982591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980091    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-cni-netd\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.982591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980117    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c611f22c-2026-4fb5-b56b-aead114a76a0-tmp-dir\") pod \"node-resolver-ptf79\" (UID: \"c611f22c-2026-4fb5-b56b-aead114a76a0\") " pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.982591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980126    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c611f22c-2026-4fb5-b56b-aead114a76a0-hosts-file\") pod \"node-resolver-ptf79\" (UID: \"c611f22c-2026-4fb5-b56b-aead114a76a0\") " pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.982591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cfb2ccd-544b-4d36-8765-e1530714db1d-ovn-node-metrics-cert\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.982591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980186    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-socket-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.982591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980191    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-cni-netd\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.982591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980217    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cfb2ccd-544b-4d36-8765-e1530714db1d-host-cni-bin\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.982591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b1d453d-626a-4d08-ad51-474bfdce1e13-cni-binary-copy\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:02.982591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.980418    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5236695-aa13-43d5-925b-e40362385900-socket-dir\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:02.983579 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.983555    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/978fd76a-8903-4505-b2f5-d8b070cbcd9a-tmp\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.983782 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.983759    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/978fd76a-8903-4505-b2f5-d8b070cbcd9a-etc-tuned\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.984093 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.983909    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cc6e861-6a80-4af5-98fa-8e19d088e530-agent-certs\") pod \"konnectivity-agent-r6qxk\" (UID: \"9cc6e861-6a80-4af5-98fa-8e19d088e530\") " pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:02.984645 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.984594    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal" event={"ID":"bf01562adb7ffda058010a76980228c7","Type":"ContainerStarted","Data":"5d5afa9ea7d79ba053cdcdefa9b5df3ca1b7e8580e7e12a1023c9fb513f430de"}
Sep 16 14:08:02.985281 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.985259    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cfb2ccd-544b-4d36-8765-e1530714db1d-ovn-node-metrics-cert\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.986579 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.986559    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq4h9\" (UniqueName: \"kubernetes.io/projected/87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349-kube-api-access-qq4h9\") pod \"multus-z8r9s\" (UID: \"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349\") " pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:02.986683 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.986664    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbvw6\" (UniqueName: \"kubernetes.io/projected/978fd76a-8903-4505-b2f5-d8b070cbcd9a-kube-api-access-tbvw6\") pod \"tuned-rvthp\" (UID: \"978fd76a-8903-4505-b2f5-d8b070cbcd9a\") " pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:02.987302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.987276    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal" event={"ID":"f4943e5a79d3f51d24a525e5e47d75f0","Type":"ContainerStarted","Data":"4d7b0146fb647e335153105f7d624d5f4e30d8ba88b77cfeb9f364d04f038828"}
Sep 16 14:08:02.987513 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.987496    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmg4k\" (UniqueName: \"kubernetes.io/projected/5cfb2ccd-544b-4d36-8765-e1530714db1d-kube-api-access-nmg4k\") pod \"ovnkube-node-mb584\" (UID: \"5cfb2ccd-544b-4d36-8765-e1530714db1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:02.991875 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.991852    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcdnf\" (UniqueName: \"kubernetes.io/projected/0d67f7d8-3265-4271-8e17-9cf0b1a6ea73-kube-api-access-jcdnf\") pod \"iptables-alerter-4dm64\" (UID: \"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73\") " pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:02.993725 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.993665    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7bql\" (UniqueName: \"kubernetes.io/projected/c611f22c-2026-4fb5-b56b-aead114a76a0-kube-api-access-v7bql\") pod \"node-resolver-ptf79\" (UID: \"c611f22c-2026-4fb5-b56b-aead114a76a0\") " pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:02.995938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:02.995917    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7ntp\" (UniqueName: \"kubernetes.io/projected/f5236695-aa13-43d5-925b-e40362385900-kube-api-access-c7ntp\") pod \"aws-ebs-csi-driver-node-m64dm\" (UID: \"f5236695-aa13-43d5-925b-e40362385900\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:03.080740 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.080687    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:03.080911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.080749    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b1d453d-626a-4d08-ad51-474bfdce1e13-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.080911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.080783    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b1d453d-626a-4d08-ad51-474bfdce1e13-system-cni-dir\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.080911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.080809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/087e2244-c0c9-4141-8f5f-b94636c32bf4-serviceca\") pod \"node-ca-kw5db\" (UID: \"087e2244-c0c9-4141-8f5f-b94636c32bf4\") " pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:03.080911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.080839    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/087e2244-c0c9-4141-8f5f-b94636c32bf4-host\") pod \"node-ca-kw5db\" (UID: \"087e2244-c0c9-4141-8f5f-b94636c32bf4\") " pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:03.080911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.080877    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp7j5\" (UniqueName: \"kubernetes.io/projected/087e2244-c0c9-4141-8f5f-b94636c32bf4-kube-api-access-wp7j5\") pod \"node-ca-kw5db\" (UID: \"087e2244-c0c9-4141-8f5f-b94636c32bf4\") " pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:03.080911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.080887    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b1d453d-626a-4d08-ad51-474bfdce1e13-system-cni-dir\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.081248 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.080907    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnz6m\" (UniqueName: \"kubernetes.io/projected/e3059f22-c414-4a85-9a55-47485f8a88b3-kube-api-access-dnz6m\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:03.081248 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.080975    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b1d453d-626a-4d08-ad51-474bfdce1e13-cnibin\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.081248 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081010    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b1d453d-626a-4d08-ad51-474bfdce1e13-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.081248 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081042    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b1d453d-626a-4d08-ad51-474bfdce1e13-cni-binary-copy\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.081248 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5szv\" (UniqueName: \"kubernetes.io/projected/5b1d453d-626a-4d08-ad51-474bfdce1e13-kube-api-access-g5szv\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.081248 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081097    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b1d453d-626a-4d08-ad51-474bfdce1e13-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.081248 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081125    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:03.081248 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081197    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/087e2244-c0c9-4141-8f5f-b94636c32bf4-host\") pod \"node-ca-kw5db\" (UID: \"087e2244-c0c9-4141-8f5f-b94636c32bf4\") " pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:03.081248 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.081234    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:03.081730 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.081292    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs podName:e3059f22-c414-4a85-9a55-47485f8a88b3 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:03.581273519 +0000 UTC m=+3.058505580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs") pod "network-metrics-daemon-sjjm9" (UID: "e3059f22-c414-4a85-9a55-47485f8a88b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:03.081730 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b1d453d-626a-4d08-ad51-474bfdce1e13-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.081730 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081422    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/087e2244-c0c9-4141-8f5f-b94636c32bf4-serviceca\") pod \"node-ca-kw5db\" (UID: \"087e2244-c0c9-4141-8f5f-b94636c32bf4\") " pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:03.081730 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081466    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b1d453d-626a-4d08-ad51-474bfdce1e13-cnibin\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.081730 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081603    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b1d453d-626a-4d08-ad51-474bfdce1e13-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.082003 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081783    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b1d453d-626a-4d08-ad51-474bfdce1e13-cni-binary-copy\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.082003 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.081920    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b1d453d-626a-4d08-ad51-474bfdce1e13-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.091630 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.091559    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:03.091630 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.091604    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:03.091630 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.091615    2566 projected.go:196] Error preparing data for projected volume kube-api-access-js55n for pod openshift-network-diagnostics/network-check-target-8v4g2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:03.091863 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.091672    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n podName:bebea01f-e7af-48bd-b462-a5cc39c387fc nodeName:}" failed. No retries permitted until 2026-09-16 14:08:03.591654304 +0000 UTC m=+3.068886377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-js55n" (UniqueName: "kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n") pod "network-check-target-8v4g2" (UID: "bebea01f-e7af-48bd-b462-a5cc39c387fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:03.094888 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.094856    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp7j5\" (UniqueName: \"kubernetes.io/projected/087e2244-c0c9-4141-8f5f-b94636c32bf4-kube-api-access-wp7j5\") pod \"node-ca-kw5db\" (UID: \"087e2244-c0c9-4141-8f5f-b94636c32bf4\") " pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:03.095086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.095065    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnz6m\" (UniqueName: \"kubernetes.io/projected/e3059f22-c414-4a85-9a55-47485f8a88b3-kube-api-access-dnz6m\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:03.097245 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.097217    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5szv\" (UniqueName: \"kubernetes.io/projected/5b1d453d-626a-4d08-ad51-474bfdce1e13-kube-api-access-g5szv\") pod \"multus-additional-cni-plugins-7lcv6\" (UID: \"5b1d453d-626a-4d08-ad51-474bfdce1e13\") " pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.160339 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.160296    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm"
Sep 16 14:08:03.170467 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.170439    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8r9s"
Sep 16 14:08:03.174360 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:03.174325    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5236695_aa13_43d5_925b_e40362385900.slice/crio-e77b0854a1bb79e481e5579502e770dd79443d19a1fa1c2d9238ec79970cd4a9 WatchSource:0}: Error finding container e77b0854a1bb79e481e5579502e770dd79443d19a1fa1c2d9238ec79970cd4a9: Status 404 returned error can't find the container with id e77b0854a1bb79e481e5579502e770dd79443d19a1fa1c2d9238ec79970cd4a9
Sep 16 14:08:03.178103 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.178064    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ptf79"
Sep 16 14:08:03.182810 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:03.182770    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87bc3c4d_27ec_4087_9e7f_a8c7ae6c3349.slice/crio-4f47bcb37302290af0bd34f593ea75e9eff8ac3991e60b79371977392ca33468 WatchSource:0}: Error finding container 4f47bcb37302290af0bd34f593ea75e9eff8ac3991e60b79371977392ca33468: Status 404 returned error can't find the container with id 4f47bcb37302290af0bd34f593ea75e9eff8ac3991e60b79371977392ca33468
Sep 16 14:08:03.185841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.185822    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:03.188374 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:03.188338    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc611f22c_2026_4fb5_b56b_aead114a76a0.slice/crio-c11cd798e2971f1acb8321e0c781408f3fab1ca72a6c83fd3ac27a1a72288113 WatchSource:0}: Error finding container c11cd798e2971f1acb8321e0c781408f3fab1ca72a6c83fd3ac27a1a72288113: Status 404 returned error can't find the container with id c11cd798e2971f1acb8321e0c781408f3fab1ca72a6c83fd3ac27a1a72288113
Sep 16 14:08:03.192752 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.192722    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:03.196170 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:03.196142    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfb2ccd_544b_4d36_8765_e1530714db1d.slice/crio-6332d27f77aab6097567e4d3a77d081cb6a9da21f6bd4939d208637d2f863b2c WatchSource:0}: Error finding container 6332d27f77aab6097567e4d3a77d081cb6a9da21f6bd4939d208637d2f863b2c: Status 404 returned error can't find the container with id 6332d27f77aab6097567e4d3a77d081cb6a9da21f6bd4939d208637d2f863b2c
Sep 16 14:08:03.200320 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.200283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4dm64"
Sep 16 14:08:03.202935 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:03.202901    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cc6e861_6a80_4af5_98fa_8e19d088e530.slice/crio-714720359d00a59db23b78cc67cd04fdc1f21bd3913b4b5996cdaf7e10749491 WatchSource:0}: Error finding container 714720359d00a59db23b78cc67cd04fdc1f21bd3913b4b5996cdaf7e10749491: Status 404 returned error can't find the container with id 714720359d00a59db23b78cc67cd04fdc1f21bd3913b4b5996cdaf7e10749491
Sep 16 14:08:03.207901 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.207740    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rvthp"
Sep 16 14:08:03.210369 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:03.210345    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d67f7d8_3265_4271_8e17_9cf0b1a6ea73.slice/crio-d6932d91650e83c0ffbf32f4e3392b1f81912fd380e1177eac0ced34ff7feb58 WatchSource:0}: Error finding container d6932d91650e83c0ffbf32f4e3392b1f81912fd380e1177eac0ced34ff7feb58: Status 404 returned error can't find the container with id d6932d91650e83c0ffbf32f4e3392b1f81912fd380e1177eac0ced34ff7feb58
Sep 16 14:08:03.215831 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.215810    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7lcv6"
Sep 16 14:08:03.218829 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:03.218811    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978fd76a_8903_4505_b2f5_d8b070cbcd9a.slice/crio-e496955dfdae7f1e7428bde4e31be88252e664ce0d47e1f688836f5551e0b7fb WatchSource:0}: Error finding container e496955dfdae7f1e7428bde4e31be88252e664ce0d47e1f688836f5551e0b7fb: Status 404 returned error can't find the container with id e496955dfdae7f1e7428bde4e31be88252e664ce0d47e1f688836f5551e0b7fb
Sep 16 14:08:03.220295 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.220276    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kw5db"
Sep 16 14:08:03.257775 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.257713    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:03.485217 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.485174    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:03.485404 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.485373    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:03.485480 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.485458    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret podName:c1593320-c5df-4f1b-bb35-b90738b70602 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:04.48543607 +0000 UTC m=+3.962668134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret") pod "global-pull-secret-syncer-97f7n" (UID: "c1593320-c5df-4f1b-bb35-b90738b70602") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:03.565563 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:03.565497    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b1d453d_626a_4d08_ad51_474bfdce1e13.slice/crio-d4266b6f78a551603c35ced373aecde9937bed9cd3c66c18df9bdb383f550475 WatchSource:0}: Error finding container d4266b6f78a551603c35ced373aecde9937bed9cd3c66c18df9bdb383f550475: Status 404 returned error can't find the container with id d4266b6f78a551603c35ced373aecde9937bed9cd3c66c18df9bdb383f550475
Sep 16 14:08:03.570815 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:03.570791    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod087e2244_c0c9_4141_8f5f_b94636c32bf4.slice/crio-43f3747d36287449d30437b5c78c88f0809aed30b79c5a2dbdfc564d7bbce5be WatchSource:0}: Error finding container 43f3747d36287449d30437b5c78c88f0809aed30b79c5a2dbdfc564d7bbce5be: Status 404 returned error can't find the container with id 43f3747d36287449d30437b5c78c88f0809aed30b79c5a2dbdfc564d7bbce5be
Sep 16 14:08:03.586065 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.586038    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:03.586227 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.586207    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:03.586345 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.586280    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs podName:e3059f22-c414-4a85-9a55-47485f8a88b3 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:04.586259388 +0000 UTC m=+4.063491452 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs") pod "network-metrics-daemon-sjjm9" (UID: "e3059f22-c414-4a85-9a55-47485f8a88b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:03.687264 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.687225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:03.687454 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.687425    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:03.687454 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.687458    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:03.687678 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.687475    2566 projected.go:196] Error preparing data for projected volume kube-api-access-js55n for pod openshift-network-diagnostics/network-check-target-8v4g2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:03.687678 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.687533    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n podName:bebea01f-e7af-48bd-b462-a5cc39c387fc nodeName:}" failed. No retries permitted until 2026-09-16 14:08:04.687510874 +0000 UTC m=+4.164742932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-js55n" (UniqueName: "kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n") pod "network-check-target-8v4g2" (UID: "bebea01f-e7af-48bd-b462-a5cc39c387fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:03.775017 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.774987    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:03.926543 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.926418    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 14:03:03 +0000 UTC" deadline="2028-03-19 17:34:45.532765665 +0000 UTC"
Sep 16 14:08:03.926543 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.926459    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13203h26m41.606310904s"
Sep 16 14:08:03.961089 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.961056    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:03.978232 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.978202    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:03.978399 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:03.978329    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:03.994440 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.994404    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" event={"ID":"5b1d453d-626a-4d08-ad51-474bfdce1e13","Type":"ContainerStarted","Data":"d4266b6f78a551603c35ced373aecde9937bed9cd3c66c18df9bdb383f550475"}
Sep 16 14:08:03.996135 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.996080    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rvthp" event={"ID":"978fd76a-8903-4505-b2f5-d8b070cbcd9a","Type":"ContainerStarted","Data":"e496955dfdae7f1e7428bde4e31be88252e664ce0d47e1f688836f5551e0b7fb"}
Sep 16 14:08:03.998152 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.997767    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4dm64" event={"ID":"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73","Type":"ContainerStarted","Data":"d6932d91650e83c0ffbf32f4e3392b1f81912fd380e1177eac0ced34ff7feb58"}
Sep 16 14:08:03.999759 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:03.999479    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r6qxk" event={"ID":"9cc6e861-6a80-4af5-98fa-8e19d088e530","Type":"ContainerStarted","Data":"714720359d00a59db23b78cc67cd04fdc1f21bd3913b4b5996cdaf7e10749491"}
Sep 16 14:08:04.001847 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.001804    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ptf79" event={"ID":"c611f22c-2026-4fb5-b56b-aead114a76a0","Type":"ContainerStarted","Data":"c11cd798e2971f1acb8321e0c781408f3fab1ca72a6c83fd3ac27a1a72288113"}
Sep 16 14:08:04.011146 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.011080    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8r9s" event={"ID":"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349","Type":"ContainerStarted","Data":"4f47bcb37302290af0bd34f593ea75e9eff8ac3991e60b79371977392ca33468"}
Sep 16 14:08:04.013052 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.012982    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm" event={"ID":"f5236695-aa13-43d5-925b-e40362385900","Type":"ContainerStarted","Data":"e77b0854a1bb79e481e5579502e770dd79443d19a1fa1c2d9238ec79970cd4a9"}
Sep 16 14:08:04.015340 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.015284    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kw5db" event={"ID":"087e2244-c0c9-4141-8f5f-b94636c32bf4","Type":"ContainerStarted","Data":"43f3747d36287449d30437b5c78c88f0809aed30b79c5a2dbdfc564d7bbce5be"}
Sep 16 14:08:04.021698 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.021673    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" event={"ID":"5cfb2ccd-544b-4d36-8765-e1530714db1d","Type":"ContainerStarted","Data":"6332d27f77aab6097567e4d3a77d081cb6a9da21f6bd4939d208637d2f863b2c"}
Sep 16 14:08:04.494844 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.494105    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:04.494844 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.494305    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:04.494844 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.494377    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret podName:c1593320-c5df-4f1b-bb35-b90738b70602 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:06.494355328 +0000 UTC m=+5.971587386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret") pod "global-pull-secret-syncer-97f7n" (UID: "c1593320-c5df-4f1b-bb35-b90738b70602") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:04.594639 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.594571    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:04.594979 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.594956    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:04.595069 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.595036    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs podName:e3059f22-c414-4a85-9a55-47485f8a88b3 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:06.595015453 +0000 UTC m=+6.072247511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs") pod "network-metrics-daemon-sjjm9" (UID: "e3059f22-c414-4a85-9a55-47485f8a88b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:04.696241 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.695605    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:04.696241 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.695774    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:04.696241 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.695810    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:04.696241 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.695823    2566 projected.go:196] Error preparing data for projected volume kube-api-access-js55n for pod openshift-network-diagnostics/network-check-target-8v4g2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:04.696241 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.695886    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n podName:bebea01f-e7af-48bd-b462-a5cc39c387fc nodeName:}" failed. No retries permitted until 2026-09-16 14:08:06.695866468 +0000 UTC m=+6.173098540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-js55n" (UniqueName: "kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n") pod "network-check-target-8v4g2" (UID: "bebea01f-e7af-48bd-b462-a5cc39c387fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:04.927186 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.927113    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 14:03:03 +0000 UTC" deadline="2028-05-02 10:27:58.677577766 +0000 UTC"
Sep 16 14:08:04.927186 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.927181    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14252h19m53.750401543s"
Sep 16 14:08:04.978076 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.978044    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:04.978240 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.978197    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:04.978673 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:04.978653    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:04.978836 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:04.978785    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:05.030995 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:05.030956    2566 generic.go:356] "Generic (PLEG): container finished" podID="f4943e5a79d3f51d24a525e5e47d75f0" containerID="42334206969b872548f6b70e691929c8761fe9d36585624caa61215163de02ac" exitCode=0
Sep 16 14:08:05.031141 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:05.031025    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal" event={"ID":"f4943e5a79d3f51d24a525e5e47d75f0","Type":"ContainerDied","Data":"42334206969b872548f6b70e691929c8761fe9d36585624caa61215163de02ac"}
Sep 16 14:08:05.978334 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:05.977906    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:05.978334 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:05.978022    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:06.512038 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:06.511449    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:06.512038 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.511644    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:06.512038 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.511706    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret podName:c1593320-c5df-4f1b-bb35-b90738b70602 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:10.51168772 +0000 UTC m=+9.988919775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret") pod "global-pull-secret-syncer-97f7n" (UID: "c1593320-c5df-4f1b-bb35-b90738b70602") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:06.612658 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:06.612621    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:06.612849 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.612774    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:06.612849 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.612847    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs podName:e3059f22-c414-4a85-9a55-47485f8a88b3 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:10.612824183 +0000 UTC m=+10.090056238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs") pod "network-metrics-daemon-sjjm9" (UID: "e3059f22-c414-4a85-9a55-47485f8a88b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:06.714550 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:06.713893    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:06.714550 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.714100    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:06.714550 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.714123    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:06.714550 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.714137    2566 projected.go:196] Error preparing data for projected volume kube-api-access-js55n for pod openshift-network-diagnostics/network-check-target-8v4g2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:06.714550 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.714194    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n podName:bebea01f-e7af-48bd-b462-a5cc39c387fc nodeName:}" failed. No retries permitted until 2026-09-16 14:08:10.714175044 +0000 UTC m=+10.191407104 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-js55n" (UniqueName: "kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n") pod "network-check-target-8v4g2" (UID: "bebea01f-e7af-48bd-b462-a5cc39c387fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:06.978330 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:06.978293    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:06.978519 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.978415    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:06.978934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:06.978744    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:06.978934 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:06.978839    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:07.977986 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:07.977947    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:07.978189 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:07.978092    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:08.982207 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:08.982161    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:08.982744 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:08.982204    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:08.982744 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:08.982316    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:08.982744 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:08.982406    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:09.978452 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:09.978379    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:09.978644 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:09.978504    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:10.547793 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:10.547756    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:10.548307 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.547922    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:10.548307 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.547988    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret podName:c1593320-c5df-4f1b-bb35-b90738b70602 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:18.547968225 +0000 UTC m=+18.025200304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret") pod "global-pull-secret-syncer-97f7n" (UID: "c1593320-c5df-4f1b-bb35-b90738b70602") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:10.649196 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:10.649158    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:10.649398 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.649371    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:10.649484 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.649444    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs podName:e3059f22-c414-4a85-9a55-47485f8a88b3 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:18.649424202 +0000 UTC m=+18.126656279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs") pod "network-metrics-daemon-sjjm9" (UID: "e3059f22-c414-4a85-9a55-47485f8a88b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:10.749987 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:10.749929    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:10.750186 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.750140    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:10.750186 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.750166    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:10.750186 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.750180    2566 projected.go:196] Error preparing data for projected volume kube-api-access-js55n for pod openshift-network-diagnostics/network-check-target-8v4g2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:10.750350 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.750245    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n podName:bebea01f-e7af-48bd-b462-a5cc39c387fc nodeName:}" failed. No retries permitted until 2026-09-16 14:08:18.750224861 +0000 UTC m=+18.227456932 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-js55n" (UniqueName: "kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n") pod "network-check-target-8v4g2" (UID: "bebea01f-e7af-48bd-b462-a5cc39c387fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:10.981231 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:10.980699    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:10.981231 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.980835    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:10.981231 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:10.980994    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:10.981231 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:10.981090    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:11.978302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:11.978243    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:11.978830 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:11.978379    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:12.980001 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:12.979738    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:12.980317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:12.979821    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:12.980317 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:12.980128    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:12.980317 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:12.980211    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:13.055126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.055031    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal" event={"ID":"f4943e5a79d3f51d24a525e5e47d75f0","Type":"ContainerStarted","Data":"c0f6b4bfe3bccc59f74f3f61062ecabcab526d35fffc70655e9af9787302f246"}
Sep 16 14:08:13.059237 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.059208    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" event={"ID":"5b1d453d-626a-4d08-ad51-474bfdce1e13","Type":"ContainerStarted","Data":"bdfe91339d8a4a83b61ca423bc6a83295e320835fb3730bec55d1171dc37b731"}
Sep 16 14:08:13.062653 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.062619    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rvthp" event={"ID":"978fd76a-8903-4505-b2f5-d8b070cbcd9a","Type":"ContainerStarted","Data":"9cb75b4005b4951cc069b622d9850a4d17eb8240f6d58a3c6eab65d3a4ed0976"}
Sep 16 14:08:13.065229 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.065204    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r6qxk" event={"ID":"9cc6e861-6a80-4af5-98fa-8e19d088e530","Type":"ContainerStarted","Data":"352a1601e198ccfc497a19187d14ac132385517fc4331d76ae3bca5a1bf8fe42"}
Sep 16 14:08:13.069056 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.069029    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm" event={"ID":"f5236695-aa13-43d5-925b-e40362385900","Type":"ContainerStarted","Data":"73e884dfe6c0070d517cef8f906fe8d4da4233737aed650a8b8d06382d8fb750"}
Sep 16 14:08:13.072470 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.072445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kw5db" event={"ID":"087e2244-c0c9-4141-8f5f-b94636c32bf4","Type":"ContainerStarted","Data":"0e1863c12ba66e939d79a180ec77f28f45e1c77faf542933f1220bca137aa20b"}
Sep 16 14:08:13.076353 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.076289    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-213.ec2.internal" podStartSLOduration=10.076272032 podStartE2EDuration="10.076272032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:13.075439338 +0000 UTC m=+12.552671444" watchObservedRunningTime="2026-09-16 14:08:13.076272032 +0000 UTC m=+12.553504113"
Sep 16 14:08:13.094969 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.094914    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r6qxk" podStartSLOduration=1.792941836 podStartE2EDuration="11.094871117s" totalImagesPullingTime="9.301929281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:13.094028526 +0000 UTC m=+12.571260601" watchObservedRunningTime="2026-09-16 14:08:13.094871117 +0000 UTC m=+12.572103198"
Sep 16 14:08:13.114072 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.114010    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rvthp" podStartSLOduration=1.787342483 podStartE2EDuration="11.1139917s" totalImagesPullingTime="9.326649217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:13.113804272 +0000 UTC m=+12.591036368" watchObservedRunningTime="2026-09-16 14:08:13.1139917 +0000 UTC m=+12.591223782"
Sep 16 14:08:13.153595 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.153537    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kw5db" podStartSLOduration=2.201409841 podStartE2EDuration="11.153518066s" totalImagesPullingTime="8.952108225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:13.153369589 +0000 UTC m=+12.630601709" watchObservedRunningTime="2026-09-16 14:08:13.153518066 +0000 UTC m=+12.630750139"
Sep 16 14:08:13.978294 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:13.978246    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:13.978472 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:13.978366    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:14.978705 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:14.978668    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:14.979199 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:14.978779    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:14.979199 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:14.979093    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:14.979199 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:14.979162    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:15.078458 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:15.078405    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4dm64" event={"ID":"0d67f7d8-3265-4271-8e17-9cf0b1a6ea73","Type":"ContainerStarted","Data":"3c1a7b9ab50eee8424787d4a5461cb3f4e3d50541261cb5354d6ff80929387a0"}
Sep 16 14:08:15.099023 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:15.098974    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4dm64" podStartSLOduration=3.781771307 podStartE2EDuration="13.098955944s" totalImagesPullingTime="9.317184637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:15.098385088 +0000 UTC m=+14.575617224" watchObservedRunningTime="2026-09-16 14:08:15.098955944 +0000 UTC m=+14.576188018"
Sep 16 14:08:15.978655 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:15.978619    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:15.978843 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:15.978753    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:16.018585 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:16.018528    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:16.019224 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:16.019201    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:16.978761 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:16.978724    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:16.978958 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:16.978738    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:16.978958 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:16.978862    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:16.978958 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:16.978948    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:17.978141 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:17.978104    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:17.978380 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:17.978245    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:18.086307 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:18.086259    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b1d453d-626a-4d08-ad51-474bfdce1e13" containerID="bdfe91339d8a4a83b61ca423bc6a83295e320835fb3730bec55d1171dc37b731" exitCode=0
Sep 16 14:08:18.086718 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:18.086320    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" event={"ID":"5b1d453d-626a-4d08-ad51-474bfdce1e13","Type":"ContainerDied","Data":"bdfe91339d8a4a83b61ca423bc6a83295e320835fb3730bec55d1171dc37b731"}
Sep 16 14:08:18.615209 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:18.615009    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:18.615385 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.615183    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:18.615385 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.615341    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret podName:c1593320-c5df-4f1b-bb35-b90738b70602 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:34.615318085 +0000 UTC m=+34.092550151 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret") pod "global-pull-secret-syncer-97f7n" (UID: "c1593320-c5df-4f1b-bb35-b90738b70602") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:18.715737 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:18.715657    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:18.715861 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.715800    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:18.715896 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.715876    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs podName:e3059f22-c414-4a85-9a55-47485f8a88b3 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:34.715844248 +0000 UTC m=+34.193076301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs") pod "network-metrics-daemon-sjjm9" (UID: "e3059f22-c414-4a85-9a55-47485f8a88b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:18.816519 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:18.816482    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:18.816695 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.816655    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:18.816695 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.816674    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:18.816695 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.816684    2566 projected.go:196] Error preparing data for projected volume kube-api-access-js55n for pod openshift-network-diagnostics/network-check-target-8v4g2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:18.816841 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.816735    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n podName:bebea01f-e7af-48bd-b462-a5cc39c387fc nodeName:}" failed. No retries permitted until 2026-09-16 14:08:34.816721793 +0000 UTC m=+34.293953845 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-js55n" (UniqueName: "kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n") pod "network-check-target-8v4g2" (UID: "bebea01f-e7af-48bd-b462-a5cc39c387fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:18.978666 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:18.978552    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:18.978666 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:18.978591    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:18.978884 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.978714    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:18.978884 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:18.978832    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:19.978649 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:19.978619    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:19.979039 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:19.978719    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:20.978689 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:20.978655    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:20.979085 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:20.978765    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:20.979085 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:20.978818    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:20.979085 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:20.978869    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:21.978001 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:21.977961    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:21.978164 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:21.978101    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:22.531798 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:22.531647    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 14:08:22.929818 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:22.929678    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T14:08:22.531780617Z","UUID":"477fb984-e88d-4e33-9ebe-625e899902d3","Handler":null,"Name":"","Endpoint":""}
Sep 16 14:08:22.931778 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:22.931758    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 14:08:22.931919 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:22.931791    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 14:08:22.978506 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:22.978480    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:22.978673 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:22.978620    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:22.978739 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:22.978672    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:22.978815 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:22.978792    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:23.098392 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.098306    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ptf79" event={"ID":"c611f22c-2026-4fb5-b56b-aead114a76a0","Type":"ContainerStarted","Data":"c672f8fb28fd45a5c0a4544925849f26fe4ddfeddcd20e927b5fd380d19c1163"}
Sep 16 14:08:23.100486 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.100122    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8r9s" event={"ID":"87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349","Type":"ContainerStarted","Data":"a5a19cf4c105cf3568999d6db076cc11499b8acb148d246783704f3fbfea44e2"}
Sep 16 14:08:23.102341 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.102318    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm" event={"ID":"f5236695-aa13-43d5-925b-e40362385900","Type":"ContainerStarted","Data":"05aba72761935a4ada6578e0c6e487e89595209ebf3132172b8e5d2764e92fe2"}
Sep 16 14:08:23.103827 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.103800    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal" event={"ID":"bf01562adb7ffda058010a76980228c7","Type":"ContainerStarted","Data":"5b818ca498bd498b8dd9f5846173f5f525fe99fab7f022827010d4869dd41db3"}
Sep 16 14:08:23.107144 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.107120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" event={"ID":"5cfb2ccd-544b-4d36-8765-e1530714db1d","Type":"ContainerStarted","Data":"5554d7211f92cd53c7c2be69a1b23f2741143c6c2db56a101a5ceff9a551f8be"}
Sep 16 14:08:23.107214 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.107197    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" event={"ID":"5cfb2ccd-544b-4d36-8765-e1530714db1d","Type":"ContainerStarted","Data":"979dfbeb5af1fe535998e8d535e56f638808aeeb5d8cd27e5c396db7da8d1bfd"}
Sep 16 14:08:23.107270 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.107231    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" event={"ID":"5cfb2ccd-544b-4d36-8765-e1530714db1d","Type":"ContainerStarted","Data":"64dd59d1544a9400ab66732301416a46854e86048a9b06fd1f1436c6422e648f"}
Sep 16 14:08:23.107270 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.107259    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" event={"ID":"5cfb2ccd-544b-4d36-8765-e1530714db1d","Type":"ContainerStarted","Data":"2fd2f0a081d8e7f1bf1a95ba6a9057d48137ecb0edff7427d5d71c5f0ddcd60d"}
Sep 16 14:08:23.107351 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.107285    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" event={"ID":"5cfb2ccd-544b-4d36-8765-e1530714db1d","Type":"ContainerStarted","Data":"19cfbd07c2663cff5639ca4b3fc231b047cad2957e55fab93f34ff4e8dd28d75"}
Sep 16 14:08:23.107351 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.107325    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" event={"ID":"5cfb2ccd-544b-4d36-8765-e1530714db1d","Type":"ContainerStarted","Data":"a2f9837ff2e2888bbef3e6f663815e423a01eaae362fa3d07f69139cd18efeb2"}
Sep 16 14:08:23.114680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.114633    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ptf79" podStartSLOduration=6.863419102 podStartE2EDuration="22.114622384s" totalImagesPullingTime="15.251203282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:23.114432485 +0000 UTC m=+22.591664587" watchObservedRunningTime="2026-09-16 14:08:23.114622384 +0000 UTC m=+22.591854455"
Sep 16 14:08:23.131118 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.131062    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z8r9s" podStartSLOduration=1.953582383 podStartE2EDuration="21.131046283s" totalImagesPullingTime="19.1774639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:23.130833639 +0000 UTC m=+22.608065732" watchObservedRunningTime="2026-09-16 14:08:23.131046283 +0000 UTC m=+22.608278361"
Sep 16 14:08:23.144319 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.144274    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-213.ec2.internal" podStartSLOduration=20.144257581 podStartE2EDuration="20.144257581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:23.143427864 +0000 UTC m=+22.620659948" watchObservedRunningTime="2026-09-16 14:08:23.144257581 +0000 UTC m=+22.621489657"
Sep 16 14:08:23.978387 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:23.978349    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:23.978834 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:23.978462    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:24.112475 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:24.112438    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm" event={"ID":"f5236695-aa13-43d5-925b-e40362385900","Type":"ContainerStarted","Data":"722392bc819688478c53f8fd7b23d5e176aac5ceb28cf27f2ee076edba2605b3"}
Sep 16 14:08:24.134231 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:24.134171    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m64dm" podStartSLOduration=3.327400941 podStartE2EDuration="23.134152047s" totalImagesPullingTime="19.806751106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:24.133173895 +0000 UTC m=+23.610405982" watchObservedRunningTime="2026-09-16 14:08:24.134152047 +0000 UTC m=+23.611384124"
Sep 16 14:08:24.357114 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:24.357031    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:24.357282 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:24.357176    2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 14:08:24.357701 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:24.357675    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r6qxk"
Sep 16 14:08:24.977787 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:24.977753    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:24.977944 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:24.977892    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:24.978006 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:24.977949    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:24.978074 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:24.978051    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:25.978558 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:25.978528    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:25.978978 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:25.978641    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:26.978685 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:26.978481    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:26.979023 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:26.978534    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:26.979023 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:26.978765    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:26.979023 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:26.978829    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:27.122636 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:27.122600    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" event={"ID":"5cfb2ccd-544b-4d36-8765-e1530714db1d","Type":"ContainerStarted","Data":"26971ebbc05453b6c273c5ec909ecbf80b1fdb8b1ff6c2304fe05d4fc40c7110"}
Sep 16 14:08:27.124176 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:27.124145    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b1d453d-626a-4d08-ad51-474bfdce1e13" containerID="b4b5efc0aace29bf795f5000dedaa5170d4942e119f4160047fd7b0117734b98" exitCode=0
Sep 16 14:08:27.124287 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:27.124192    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" event={"ID":"5b1d453d-626a-4d08-ad51-474bfdce1e13","Type":"ContainerDied","Data":"b4b5efc0aace29bf795f5000dedaa5170d4942e119f4160047fd7b0117734b98"}
Sep 16 14:08:27.977654 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:27.977626    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:27.977772 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:27.977721    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:28.129101 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:28.129011    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b1d453d-626a-4d08-ad51-474bfdce1e13" containerID="95d313bc2b8e05ff477d70b9565b992eb30c93e8eb7f91401bd34270079281cf" exitCode=0
Sep 16 14:08:28.129101 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:28.129062    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" event={"ID":"5b1d453d-626a-4d08-ad51-474bfdce1e13","Type":"ContainerDied","Data":"95d313bc2b8e05ff477d70b9565b992eb30c93e8eb7f91401bd34270079281cf"}
Sep 16 14:08:28.978764 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:28.978687    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:28.978931 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:28.978789    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:28.978931 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:28.978843    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:28.978998 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:28.978933    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:29.134018 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:29.133986    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" event={"ID":"5cfb2ccd-544b-4d36-8765-e1530714db1d","Type":"ContainerStarted","Data":"955935ef81c51e33d5f39659633e53c87e3e8f7d737702effc33d5ce67fc3619"}
Sep 16 14:08:29.134611 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:29.134214    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:29.149951 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:29.149930    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:29.166534 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:29.166495    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mb584" podStartSLOduration=8.001432301 podStartE2EDuration="27.166481064s" totalImagesPullingTime="19.165048763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:29.16645976 +0000 UTC m=+28.643691854" watchObservedRunningTime="2026-09-16 14:08:29.166481064 +0000 UTC m=+28.643713139"
Sep 16 14:08:29.977980 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:29.977952    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:29.978139 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:29.978051    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:29.981007 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:29.980983    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:30.136770 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:30.136728    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:30.151708 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:30.151685    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:08:30.521096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:30.520869    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sjjm9"]
Sep 16 14:08:30.521232 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:30.521218    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:30.521366 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:30.521344    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:30.524499 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:30.524472    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8v4g2"]
Sep 16 14:08:30.524630 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:30.524606    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:30.524724 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:30.524704    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:30.527070 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:30.527048    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-97f7n"]
Sep 16 14:08:30.527185 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:30.527154    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:30.527266 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:30.527248    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:31.141704 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:31.141674    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b1d453d-626a-4d08-ad51-474bfdce1e13" containerID="cee8f1f7273517e0fe11684113c4ff5d6b3221d41a604eaee41d03c2b2e8a30a" exitCode=0
Sep 16 14:08:31.142117 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:31.141744    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" event={"ID":"5b1d453d-626a-4d08-ad51-474bfdce1e13","Type":"ContainerDied","Data":"cee8f1f7273517e0fe11684113c4ff5d6b3221d41a604eaee41d03c2b2e8a30a"}
Sep 16 14:08:31.977802 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:31.977764    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:31.977802 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:31.977804    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:31.978001 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:31.977882    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:31.978001 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:31.977909    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:31.978001 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:31.977962    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:31.978103 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:31.978038    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:33.978137 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:33.978110    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:33.978470 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:33.978111    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:33.978470 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:33.978206    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjjm9" podUID="e3059f22-c414-4a85-9a55-47485f8a88b3"
Sep 16 14:08:33.978470 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:33.978111    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:33.978470 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:33.978277    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8v4g2" podUID="bebea01f-e7af-48bd-b462-a5cc39c387fc"
Sep 16 14:08:33.978470 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:33.978339    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97f7n" podUID="c1593320-c5df-4f1b-bb35-b90738b70602"
Sep 16 14:08:34.171012 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.170938    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-213.ec2.internal" event="NodeReady"
Sep 16 14:08:34.171154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.171044    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 14:08:34.226277 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.226241    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9dtll"]
Sep 16 14:08:34.244595 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.244554    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mkm4b"]
Sep 16 14:08:34.244733 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.244651    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.248512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.248489    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 14:08:34.248665 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.248521    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 14:08:34.248665 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.248527    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8lck4\""
Sep 16 14:08:34.268850 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.268826    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9dtll"]
Sep 16 14:08:34.268953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.268866    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mkm4b"]
Sep 16 14:08:34.268953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.268928    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mkm4b"
Sep 16 14:08:34.273274 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.273255    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 14:08:34.273615 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.273594    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8lnxj\""
Sep 16 14:08:34.273615 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.273611    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 14:08:34.273782 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.273597    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 14:08:34.322883 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.322853    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncdw7\" (UniqueName: \"kubernetes.io/projected/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-kube-api-access-ncdw7\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.322883 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.322885    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-tmp-dir\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.323035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.322917    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-config-volume\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.323035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.322982    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.423263 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.423206    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.423263 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.423238    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncdw7\" (UniqueName: \"kubernetes.io/projected/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-kube-api-access-ncdw7\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.423263 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.423260    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-tmp-dir\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.423416 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.423372    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:34.423416 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.423386    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khp8x\" (UniqueName: \"kubernetes.io/projected/e821d775-7157-4923-bb4f-e5a650d51f51-kube-api-access-khp8x\") pod \"ingress-canary-mkm4b\" (UID: \"e821d775-7157-4923-bb4f-e5a650d51f51\") " pod="openshift-ingress-canary/ingress-canary-mkm4b"
Sep 16 14:08:34.423485 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.423425    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-config-volume\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.423485 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.423444    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls podName:fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:34.923425434 +0000 UTC m=+34.400657501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls") pod "dns-default-9dtll" (UID: "fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:34.423561 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.423488    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e821d775-7157-4923-bb4f-e5a650d51f51-cert\") pod \"ingress-canary-mkm4b\" (UID: \"e821d775-7157-4923-bb4f-e5a650d51f51\") " pod="openshift-ingress-canary/ingress-canary-mkm4b"
Sep 16 14:08:34.423745 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.423730    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-tmp-dir\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.423926 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.423911    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-config-volume\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.434299 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.434276    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncdw7\" (UniqueName: \"kubernetes.io/projected/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-kube-api-access-ncdw7\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.524801 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.524766    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khp8x\" (UniqueName: \"kubernetes.io/projected/e821d775-7157-4923-bb4f-e5a650d51f51-kube-api-access-khp8x\") pod \"ingress-canary-mkm4b\" (UID: \"e821d775-7157-4923-bb4f-e5a650d51f51\") " pod="openshift-ingress-canary/ingress-canary-mkm4b"
Sep 16 14:08:34.524801 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.524805    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e821d775-7157-4923-bb4f-e5a650d51f51-cert\") pod \"ingress-canary-mkm4b\" (UID: \"e821d775-7157-4923-bb4f-e5a650d51f51\") " pod="openshift-ingress-canary/ingress-canary-mkm4b"
Sep 16 14:08:34.527311 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.527293    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e821d775-7157-4923-bb4f-e5a650d51f51-cert\") pod \"ingress-canary-mkm4b\" (UID: \"e821d775-7157-4923-bb4f-e5a650d51f51\") " pod="openshift-ingress-canary/ingress-canary-mkm4b"
Sep 16 14:08:34.536001 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.535974    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khp8x\" (UniqueName: \"kubernetes.io/projected/e821d775-7157-4923-bb4f-e5a650d51f51-kube-api-access-khp8x\") pod \"ingress-canary-mkm4b\" (UID: \"e821d775-7157-4923-bb4f-e5a650d51f51\") " pod="openshift-ingress-canary/ingress-canary-mkm4b"
Sep 16 14:08:34.577075 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.577048    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mkm4b"
Sep 16 14:08:34.625719 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.625692    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:34.625859 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.625841    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:34.625917 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.625908    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret podName:c1593320-c5df-4f1b-bb35-b90738b70602 nodeName:}" failed. No retries permitted until 2026-09-16 14:09:06.625893105 +0000 UTC m=+66.103125158 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret") pod "global-pull-secret-syncer-97f7n" (UID: "c1593320-c5df-4f1b-bb35-b90738b70602") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:34.727277 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.727021    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:34.727400 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.727177    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:34.727400 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.727390    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs podName:e3059f22-c414-4a85-9a55-47485f8a88b3 nodeName:}" failed. No retries permitted until 2026-09-16 14:09:06.727370195 +0000 UTC m=+66.204602248 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs") pod "network-metrics-daemon-sjjm9" (UID: "e3059f22-c414-4a85-9a55-47485f8a88b3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:34.828500 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.828460    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:34.828658 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.828614    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:34.828658 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.828630    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:34.828658 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.828640    2566 projected.go:196] Error preparing data for projected volume kube-api-access-js55n for pod openshift-network-diagnostics/network-check-target-8v4g2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:34.828762 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.828695    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n podName:bebea01f-e7af-48bd-b462-a5cc39c387fc nodeName:}" failed. No retries permitted until 2026-09-16 14:09:06.828682455 +0000 UTC m=+66.305914507 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-js55n" (UniqueName: "kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n") pod "network-check-target-8v4g2" (UID: "bebea01f-e7af-48bd-b462-a5cc39c387fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:34.904590 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.904494    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mkm4b"]
Sep 16 14:08:34.907742 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:34.907706    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode821d775_7157_4923_bb4f_e5a650d51f51.slice/crio-ef4104e778c556567b29d63e075e2b78df4971ffb1488e82c8a6b724536f2655 WatchSource:0}: Error finding container ef4104e778c556567b29d63e075e2b78df4971ffb1488e82c8a6b724536f2655: Status 404 returned error can't find the container with id ef4104e778c556567b29d63e075e2b78df4971ffb1488e82c8a6b724536f2655
Sep 16 14:08:34.928762 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:34.928743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:34.928851 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.928839    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:34.928886 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:34.928881    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls podName:fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:35.928868469 +0000 UTC m=+35.406100522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls") pod "dns-default-9dtll" (UID: "fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:35.150961 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.150857    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mkm4b" event={"ID":"e821d775-7157-4923-bb4f-e5a650d51f51","Type":"ContainerStarted","Data":"ef4104e778c556567b29d63e075e2b78df4971ffb1488e82c8a6b724536f2655"}
Sep 16 14:08:35.607543 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.607460    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"]
Sep 16 14:08:35.631026 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.630995    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"]
Sep 16 14:08:35.631207 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.631126    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:35.636963 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.636937    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 16 14:08:35.637073 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.636971    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dkwr7\""
Sep 16 14:08:35.637455 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.637436    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 16 14:08:35.733621 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.733594    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/39204b25-1484-48aa-beff-c86bc7e67b1c-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:35.733621 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.733624    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:35.834925 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.834422    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/39204b25-1484-48aa-beff-c86bc7e67b1c-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:35.834925 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.834467    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:35.834925 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:35.834615    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 14:08:35.834925 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:35.834704    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert podName:39204b25-1484-48aa-beff-c86bc7e67b1c nodeName:}" failed. No retries permitted until 2026-09-16 14:08:36.334684987 +0000 UTC m=+35.811917044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xzgq8" (UID: "39204b25-1484-48aa-beff-c86bc7e67b1c") : secret "networking-console-plugin-cert" not found
Sep 16 14:08:35.835247 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.835045    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/39204b25-1484-48aa-beff-c86bc7e67b1c-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:35.935870 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.935784    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:35.936034 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:35.936001    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:35.936084 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:35.936060    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls podName:fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:37.936042617 +0000 UTC m=+37.413274672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls") pod "dns-default-9dtll" (UID: "fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:35.977936 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.977901    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:08:35.977936 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.977936    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:08:35.978157 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.977937    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:08:35.984397 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.984373    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 14:08:35.984524 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.984374    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x4gh8\""
Sep 16 14:08:35.984852 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.984640    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 14:08:35.985266 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.985246    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8mpwg\""
Sep 16 14:08:35.985439 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.985425    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 14:08:35.986932 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:35.986916    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 14:08:36.339179 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.339113    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:36.339489 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:36.339291    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 14:08:36.339489 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:36.339372    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert podName:39204b25-1484-48aa-beff-c86bc7e67b1c nodeName:}" failed. No retries permitted until 2026-09-16 14:08:37.33935068 +0000 UTC m=+36.816582735 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xzgq8" (UID: "39204b25-1484-48aa-beff-c86bc7e67b1c") : secret "networking-console-plugin-cert" not found
Sep 16 14:08:36.768299 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.766607    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cbwh6"]
Sep 16 14:08:36.793601 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.793546    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cbwh6"]
Sep 16 14:08:36.793762 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.793661    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:36.795936 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.795910    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 14:08:36.797021 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.797000    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 14:08:36.797112 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.797004    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 14:08:36.797229 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.797181    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 14:08:36.797815 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.797794    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c7jvw\""
Sep 16 14:08:36.943323 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.943285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:36.943323 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.943322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:36.943585 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.943367    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b484d2-5062-4a43-bd41-cf88d08a6594-data-volume\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:36.943585 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.943479    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxstz\" (UniqueName: \"kubernetes.io/projected/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-api-access-kxstz\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:36.943585 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:36.943513    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29b484d2-5062-4a43-bd41-cf88d08a6594-crio-socket\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.044730 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.044643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxstz\" (UniqueName: \"kubernetes.io/projected/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-api-access-kxstz\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.044730 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.044690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29b484d2-5062-4a43-bd41-cf88d08a6594-crio-socket\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.044927 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.044793    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.044927 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.044819    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.044927 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.044860    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b484d2-5062-4a43-bd41-cf88d08a6594-data-volume\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.045100 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.045076    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:08:37.045168 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.045156    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm podName:29b484d2-5062-4a43-bd41-cf88d08a6594 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:37.545135208 +0000 UTC m=+37.022367265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbwh6" (UID: "29b484d2-5062-4a43-bd41-cf88d08a6594") : configmap "kube-rbac-proxy" not found
Sep 16 14:08:37.045224 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.045189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b484d2-5062-4a43-bd41-cf88d08a6594-data-volume\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.045276 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.045267    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:37.045321 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.045301    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls podName:29b484d2-5062-4a43-bd41-cf88d08a6594 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:37.545290805 +0000 UTC m=+37.022522860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbwh6" (UID: "29b484d2-5062-4a43-bd41-cf88d08a6594") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:37.045477 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.045454    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29b484d2-5062-4a43-bd41-cf88d08a6594-crio-socket\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.070955 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.070920    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxstz\" (UniqueName: \"kubernetes.io/projected/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-api-access-kxstz\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.348071 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.347995    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:37.348546 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.348154    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 14:08:37.348546 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.348229    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert podName:39204b25-1484-48aa-beff-c86bc7e67b1c nodeName:}" failed. No retries permitted until 2026-09-16 14:08:39.348208343 +0000 UTC m=+38.825440401 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xzgq8" (UID: "39204b25-1484-48aa-beff-c86bc7e67b1c") : secret "networking-console-plugin-cert" not found
Sep 16 14:08:37.549821 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.549798    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.549934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.549835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:37.549934 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.549926    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:08:37.550026 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.549991    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm podName:29b484d2-5062-4a43-bd41-cf88d08a6594 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:38.549973973 +0000 UTC m=+38.027206027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbwh6" (UID: "29b484d2-5062-4a43-bd41-cf88d08a6594") : configmap "kube-rbac-proxy" not found
Sep 16 14:08:37.550026 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.550019    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:37.550136 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.550070    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls podName:29b484d2-5062-4a43-bd41-cf88d08a6594 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:38.55005451 +0000 UTC m=+38.027286572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbwh6" (UID: "29b484d2-5062-4a43-bd41-cf88d08a6594") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:37.850424 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.850390    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh"]
Sep 16 14:08:37.874075 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.873994    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh"]
Sep 16 14:08:37.874204 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.874119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh"
Sep 16 14:08:37.882146 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.881886    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ldtb2\""
Sep 16 14:08:37.882229 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.882176    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 16 14:08:37.883922 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.883900    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 16 14:08:37.953455 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.953418    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kghf9\" (UniqueName: \"kubernetes.io/projected/c00271b9-8282-4d7f-9b5a-7f27d8a3d49b-kube-api-access-kghf9\") pod \"migrator-8b4d697c5-nmdvh\" (UID: \"c00271b9-8282-4d7f-9b5a-7f27d8a3d49b\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh"
Sep 16 14:08:37.953630 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:37.953481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:37.953630 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.953610    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:37.953753 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:37.953677    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls podName:fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:41.953659063 +0000 UTC m=+41.430891122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls") pod "dns-default-9dtll" (UID: "fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:38.054534 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:38.054499    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kghf9\" (UniqueName: \"kubernetes.io/projected/c00271b9-8282-4d7f-9b5a-7f27d8a3d49b-kube-api-access-kghf9\") pod \"migrator-8b4d697c5-nmdvh\" (UID: \"c00271b9-8282-4d7f-9b5a-7f27d8a3d49b\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh"
Sep 16 14:08:38.064323 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:38.064292    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kghf9\" (UniqueName: \"kubernetes.io/projected/c00271b9-8282-4d7f-9b5a-7f27d8a3d49b-kube-api-access-kghf9\") pod \"migrator-8b4d697c5-nmdvh\" (UID: \"c00271b9-8282-4d7f-9b5a-7f27d8a3d49b\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh"
Sep 16 14:08:38.162038 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:38.161944    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mkm4b" event={"ID":"e821d775-7157-4923-bb4f-e5a650d51f51","Type":"ContainerStarted","Data":"21228aa232c02e3e9bb10c8cf7b88874445e66518324c984f53a1a511738e81b"}
Sep 16 14:08:38.180323 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:38.180260    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mkm4b" podStartSLOduration=1.550277398 podStartE2EDuration="4.180240209s" totalImagesPullingTime="2.629962811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:38.179662378 +0000 UTC m=+37.656894474" watchObservedRunningTime="2026-09-16 14:08:38.180240209 +0000 UTC m=+37.657472277"
Sep 16 14:08:38.185665 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:38.185634    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh"
Sep 16 14:08:38.481754 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:38.481530    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh"]
Sep 16 14:08:38.557988 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:38.557950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:38.557988 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:38.557987    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:38.558200 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:38.558104    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:08:38.558200 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:38.558136    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:38.558200 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:38.558184    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm podName:29b484d2-5062-4a43-bd41-cf88d08a6594 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:40.558163413 +0000 UTC m=+40.035395482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbwh6" (UID: "29b484d2-5062-4a43-bd41-cf88d08a6594") : configmap "kube-rbac-proxy" not found
Sep 16 14:08:38.558337 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:38.558207    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls podName:29b484d2-5062-4a43-bd41-cf88d08a6594 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:40.558194588 +0000 UTC m=+40.035426645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbwh6" (UID: "29b484d2-5062-4a43-bd41-cf88d08a6594") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:38.910699 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:38.910599    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ptf79_c611f22c-2026-4fb5-b56b-aead114a76a0/dns-node-resolver/0.log"
Sep 16 14:08:39.221915 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.221885    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-zbwbd"]
Sep 16 14:08:39.276509 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.276471    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-zbwbd"]
Sep 16 14:08:39.276685 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.276605    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.280069 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.280046    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 16 14:08:39.280200 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.280048    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 16 14:08:39.280817 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.280795    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 16 14:08:39.280918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.280804    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vkhbh\""
Sep 16 14:08:39.280918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.280805    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 16 14:08:39.364514 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.364462    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9l6h\" (UniqueName: \"kubernetes.io/projected/6f284702-4ff9-4d3e-9231-75292724a5f5-kube-api-access-c9l6h\") pod \"service-ca-568869fdc5-zbwbd\" (UID: \"6f284702-4ff9-4d3e-9231-75292724a5f5\") " pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.364766 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.364641    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6f284702-4ff9-4d3e-9231-75292724a5f5-signing-cabundle\") pod \"service-ca-568869fdc5-zbwbd\" (UID: \"6f284702-4ff9-4d3e-9231-75292724a5f5\") " pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.364766 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.364671    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6f284702-4ff9-4d3e-9231-75292724a5f5-signing-key\") pod \"service-ca-568869fdc5-zbwbd\" (UID: \"6f284702-4ff9-4d3e-9231-75292724a5f5\") " pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.364929 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.364806    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:39.365036 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:39.364961    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 14:08:39.365036 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:39.365015    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert podName:39204b25-1484-48aa-beff-c86bc7e67b1c nodeName:}" failed. No retries permitted until 2026-09-16 14:08:43.365001764 +0000 UTC m=+42.842233817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xzgq8" (UID: "39204b25-1484-48aa-beff-c86bc7e67b1c") : secret "networking-console-plugin-cert" not found
Sep 16 14:08:39.465409 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.465365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9l6h\" (UniqueName: \"kubernetes.io/projected/6f284702-4ff9-4d3e-9231-75292724a5f5-kube-api-access-c9l6h\") pod \"service-ca-568869fdc5-zbwbd\" (UID: \"6f284702-4ff9-4d3e-9231-75292724a5f5\") " pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.465608 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.465462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6f284702-4ff9-4d3e-9231-75292724a5f5-signing-cabundle\") pod \"service-ca-568869fdc5-zbwbd\" (UID: \"6f284702-4ff9-4d3e-9231-75292724a5f5\") " pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.465608 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.465490    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6f284702-4ff9-4d3e-9231-75292724a5f5-signing-key\") pod \"service-ca-568869fdc5-zbwbd\" (UID: \"6f284702-4ff9-4d3e-9231-75292724a5f5\") " pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.466344 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.466310    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6f284702-4ff9-4d3e-9231-75292724a5f5-signing-cabundle\") pod \"service-ca-568869fdc5-zbwbd\" (UID: \"6f284702-4ff9-4d3e-9231-75292724a5f5\") " pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.468943 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.468922    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6f284702-4ff9-4d3e-9231-75292724a5f5-signing-key\") pod \"service-ca-568869fdc5-zbwbd\" (UID: \"6f284702-4ff9-4d3e-9231-75292724a5f5\") " pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.475424 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.475374    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9l6h\" (UniqueName: \"kubernetes.io/projected/6f284702-4ff9-4d3e-9231-75292724a5f5-kube-api-access-c9l6h\") pod \"service-ca-568869fdc5-zbwbd\" (UID: \"6f284702-4ff9-4d3e-9231-75292724a5f5\") " pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.587740 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.587705    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-zbwbd"
Sep 16 14:08:39.710426 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:39.710395    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kw5db_087e2244-c0c9-4141-8f5f-b94636c32bf4/node-ca/0.log"
Sep 16 14:08:39.869822 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:39.869744    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00271b9_8282_4d7f_9b5a_7f27d8a3d49b.slice/crio-960ebb91f69694eed8be7accd5f16b6446d9ffb0dfb5f0c06611a5cc8a95065f WatchSource:0}: Error finding container 960ebb91f69694eed8be7accd5f16b6446d9ffb0dfb5f0c06611a5cc8a95065f: Status 404 returned error can't find the container with id 960ebb91f69694eed8be7accd5f16b6446d9ffb0dfb5f0c06611a5cc8a95065f
Sep 16 14:08:40.139335 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:40.139228    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-zbwbd"]
Sep 16 14:08:40.168359 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:40.168333    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh" event={"ID":"c00271b9-8282-4d7f-9b5a-7f27d8a3d49b","Type":"ContainerStarted","Data":"960ebb91f69694eed8be7accd5f16b6446d9ffb0dfb5f0c06611a5cc8a95065f"}
Sep 16 14:08:40.184316 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:40.184291    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f284702_4ff9_4d3e_9231_75292724a5f5.slice/crio-4c7016ca65ea7f58fe2a9da5af04eb188757e61dfa93eddb81c7a51d644eef70 WatchSource:0}: Error finding container 4c7016ca65ea7f58fe2a9da5af04eb188757e61dfa93eddb81c7a51d644eef70: Status 404 returned error can't find the container with id 4c7016ca65ea7f58fe2a9da5af04eb188757e61dfa93eddb81c7a51d644eef70
Sep 16 14:08:40.575209 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:40.575173    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:40.575411 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:40.575212    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:40.575411 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:40.575303    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:08:40.575411 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:40.575375    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm podName:29b484d2-5062-4a43-bd41-cf88d08a6594 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:44.575359104 +0000 UTC m=+44.052591162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cbwh6" (UID: "29b484d2-5062-4a43-bd41-cf88d08a6594") : configmap "kube-rbac-proxy" not found
Sep 16 14:08:40.575411 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:40.575383    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:40.575608 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:40.575436    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls podName:29b484d2-5062-4a43-bd41-cf88d08a6594 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:44.575419455 +0000 UTC m=+44.052651510 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbwh6" (UID: "29b484d2-5062-4a43-bd41-cf88d08a6594") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:40.710722 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:40.710682    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mkm4b_e821d775-7157-4923-bb4f-e5a650d51f51/serve-healthcheck-canary/0.log"
Sep 16 14:08:41.173088 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:41.173002    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-zbwbd" event={"ID":"6f284702-4ff9-4d3e-9231-75292724a5f5","Type":"ContainerStarted","Data":"4c7016ca65ea7f58fe2a9da5af04eb188757e61dfa93eddb81c7a51d644eef70"}
Sep 16 14:08:41.176218 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:41.176184    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b1d453d-626a-4d08-ad51-474bfdce1e13" containerID="35cf63d9883a9d06130887c363b3043f16b5f8742cc804c15ba0a689f11ce572" exitCode=0
Sep 16 14:08:41.176360 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:41.176232    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" event={"ID":"5b1d453d-626a-4d08-ad51-474bfdce1e13","Type":"ContainerDied","Data":"35cf63d9883a9d06130887c363b3043f16b5f8742cc804c15ba0a689f11ce572"}
Sep 16 14:08:41.985835 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:41.985810    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:41.986148 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:41.985910    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:41.986148 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:41.985960    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls podName:fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:49.985947782 +0000 UTC m=+49.463179834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls") pod "dns-default-9dtll" (UID: "fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:42.180767 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:42.180501    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh" event={"ID":"c00271b9-8282-4d7f-9b5a-7f27d8a3d49b","Type":"ContainerStarted","Data":"2eb279023c896e8ce2b17fa6a7a4171b7820e601a1101acfad7f9adb3df91087"}
Sep 16 14:08:42.180953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:42.180831    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh" event={"ID":"c00271b9-8282-4d7f-9b5a-7f27d8a3d49b","Type":"ContainerStarted","Data":"42a2c78ffbe097f5681b502a8463b4bc14d7c8d8aed0c8eee33105441d763045"}
Sep 16 14:08:42.183602 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:42.183559    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b1d453d-626a-4d08-ad51-474bfdce1e13" containerID="316550508fa79167bf403a065fe746ec6d65fc511912512ecf4ee31e073dd3c0" exitCode=0
Sep 16 14:08:42.183602 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:42.183596    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" event={"ID":"5b1d453d-626a-4d08-ad51-474bfdce1e13","Type":"ContainerDied","Data":"316550508fa79167bf403a065fe746ec6d65fc511912512ecf4ee31e073dd3c0"}
Sep 16 14:08:42.196953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:42.196916    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-nmdvh" podStartSLOduration=3.189743918 podStartE2EDuration="5.196902182s" totalImagesPullingTime="2.007158264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:42.196164024 +0000 UTC m=+41.673396139" watchObservedRunningTime="2026-09-16 14:08:42.196902182 +0000 UTC m=+41.674134257"
Sep 16 14:08:43.189940 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:43.189903    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" event={"ID":"5b1d453d-626a-4d08-ad51-474bfdce1e13","Type":"ContainerStarted","Data":"f2afcde3d63cde302ba71b993899e34eebdea466241330b2f4a572e54bfb1dd0"}
Sep 16 14:08:43.213046 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:43.212996    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7lcv6" podStartSLOduration=11.679549565 podStartE2EDuration="41.21297918s" totalImagesPullingTime="29.533429615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:43.211111453 +0000 UTC m=+42.688343541" watchObservedRunningTime="2026-09-16 14:08:43.21297918 +0000 UTC m=+42.690211255"
Sep 16 14:08:43.396711 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:43.396676    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:43.396872 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:43.396848    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 14:08:43.396947 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:43.396936    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert podName:39204b25-1484-48aa-beff-c86bc7e67b1c nodeName:}" failed. No retries permitted until 2026-09-16 14:08:51.396913381 +0000 UTC m=+50.874145449 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xzgq8" (UID: "39204b25-1484-48aa-beff-c86bc7e67b1c") : secret "networking-console-plugin-cert" not found
Sep 16 14:08:44.194012 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:44.193968    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-zbwbd" event={"ID":"6f284702-4ff9-4d3e-9231-75292724a5f5","Type":"ContainerStarted","Data":"24ee5e669f1e3b4314a7f1e83a2d698f50997abc539387c74dd5ee4c4da0cdb5"}
Sep 16 14:08:44.211147 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:44.211103    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-zbwbd" podStartSLOduration=2.146968294 podStartE2EDuration="5.211078329s" totalImagesPullingTime="3.064110035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:44.210972439 +0000 UTC m=+43.688204535" watchObservedRunningTime="2026-09-16 14:08:44.211078329 +0000 UTC m=+43.688310403"
Sep 16 14:08:44.605007 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:44.604904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:44.605007 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:44.604956    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:44.605254 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:44.605102    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:44.605254 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:44.605171    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls podName:29b484d2-5062-4a43-bd41-cf88d08a6594 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:52.605152565 +0000 UTC m=+52.082384620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cbwh6" (UID: "29b484d2-5062-4a43-bd41-cf88d08a6594") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:44.605628 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:44.605600    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b484d2-5062-4a43-bd41-cf88d08a6594-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:50.045576 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:50.045532    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:50.049234 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:50.049214    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0-metrics-tls\") pod \"dns-default-9dtll\" (UID: \"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0\") " pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:50.153775 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:50.153750    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:50.460280 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:50.460240    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9dtll"]
Sep 16 14:08:50.469274 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:50.469132    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe96b29b_d87b_4d2f_b24f_f1e8fb1573e0.slice/crio-eff744a3256608a58de21f9c5fca9cac93cb44bde6bd72b9ad2c15404db71678 WatchSource:0}: Error finding container eff744a3256608a58de21f9c5fca9cac93cb44bde6bd72b9ad2c15404db71678: Status 404 returned error can't find the container with id eff744a3256608a58de21f9c5fca9cac93cb44bde6bd72b9ad2c15404db71678
Sep 16 14:08:51.213325 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:51.213284    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9dtll" event={"ID":"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0","Type":"ContainerStarted","Data":"eff744a3256608a58de21f9c5fca9cac93cb44bde6bd72b9ad2c15404db71678"}
Sep 16 14:08:51.457555 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:51.457512    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:08:51.457753 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:51.457688    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 14:08:51.457810 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:08:51.457760    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert podName:39204b25-1484-48aa-beff-c86bc7e67b1c nodeName:}" failed. No retries permitted until 2026-09-16 14:09:07.457743429 +0000 UTC m=+66.934975486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xzgq8" (UID: "39204b25-1484-48aa-beff-c86bc7e67b1c") : secret "networking-console-plugin-cert" not found
Sep 16 14:08:52.668186 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:52.668144    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:52.670875 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:52.670851    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b484d2-5062-4a43-bd41-cf88d08a6594-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cbwh6\" (UID: \"29b484d2-5062-4a43-bd41-cf88d08a6594\") " pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:52.706605 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:52.706555    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cbwh6"
Sep 16 14:08:52.991275 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:52.991241    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cbwh6"]
Sep 16 14:08:52.995258 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:08:52.995225    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b484d2_5062_4a43_bd41_cf88d08a6594.slice/crio-c061617918708fb720d351a3a5a7aaa92a93496d5f268c94096b109e9bc39aff WatchSource:0}: Error finding container c061617918708fb720d351a3a5a7aaa92a93496d5f268c94096b109e9bc39aff: Status 404 returned error can't find the container with id c061617918708fb720d351a3a5a7aaa92a93496d5f268c94096b109e9bc39aff
Sep 16 14:08:53.220984 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:53.220880    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbwh6" event={"ID":"29b484d2-5062-4a43-bd41-cf88d08a6594","Type":"ContainerStarted","Data":"25d87b07636fe44254fb78451aadb82d42094efe25f5de41ad9abc7d7a248918"}
Sep 16 14:08:53.220984 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:53.220954    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbwh6" event={"ID":"29b484d2-5062-4a43-bd41-cf88d08a6594","Type":"ContainerStarted","Data":"c061617918708fb720d351a3a5a7aaa92a93496d5f268c94096b109e9bc39aff"}
Sep 16 14:08:54.227359 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:54.227321    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9dtll" event={"ID":"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0","Type":"ContainerStarted","Data":"f5fd1fdba606831ef8971f09a6560d1449a3e3ef741e73f37f046957d94c66e3"}
Sep 16 14:08:54.227828 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:54.227375    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9dtll" event={"ID":"fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0","Type":"ContainerStarted","Data":"83bc419dd5e39bb3d0fee175a4833e35786d5b8ede19e1dc68849df194e52293"}
Sep 16 14:08:54.227828 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:54.227512    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9dtll"
Sep 16 14:08:54.229140 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:54.229116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbwh6" event={"ID":"29b484d2-5062-4a43-bd41-cf88d08a6594","Type":"ContainerStarted","Data":"9d82841a543a1d96086e3863644dce4b906f25888ea8984a7f572b3c48940def"}
Sep 16 14:08:54.247776 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:54.247740    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9dtll" podStartSLOduration=17.100264545 podStartE2EDuration="20.247727454s" totalImagesPullingTime="3.147462909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:54.247002614 +0000 UTC m=+53.724234689" watchObservedRunningTime="2026-09-16 14:08:54.247727454 +0000 UTC m=+53.724959526"
Sep 16 14:08:56.239885 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:56.239852    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cbwh6" event={"ID":"29b484d2-5062-4a43-bd41-cf88d08a6594","Type":"ContainerStarted","Data":"9b2833bdf000a5f03ea8102ecc50b8bfdbc85893b944c5989977e5eef2c96de5"}
Sep 16 14:08:56.262968 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:56.262900    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cbwh6" podStartSLOduration=17.497899694 podStartE2EDuration="20.262880823s" totalImagesPullingTime="2.764981129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:56.262845881 +0000 UTC m=+55.740077969" watchObservedRunningTime="2026-09-16 14:08:56.262880823 +0000 UTC m=+55.740112924"
Sep 16 14:08:59.466810 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.466774    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"]
Sep 16 14:08:59.470100 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.470082    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"]
Sep 16 14:08:59.470218 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.470202    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.472112 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.472094    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"
Sep 16 14:08:59.476710 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.474730    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 14:08:59.476710 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.474734    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 14:08:59.476710 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.474997    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bppxs\""
Sep 16 14:08:59.476710 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.475270    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 16 14:08:59.476710 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.475483    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 14:08:59.476710 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.475710    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 16 14:08:59.492541 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.492516    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"]
Sep 16 14:08:59.493267 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.493247    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"]
Sep 16 14:08:59.560763 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.560730    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-g67hh"]
Sep 16 14:08:59.562691 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.562673    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-g67hh"
Sep 16 14:08:59.567326 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.567300    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 14:08:59.567606 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.567588    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4qsq9\""
Sep 16 14:08:59.567886 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.567862    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 14:08:59.581904 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.581878    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-g67hh"]
Sep 16 14:08:59.623309 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.623285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e618803-f23d-49c6-896d-f5095313e542-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-586c5cdf58-cfb9j\" (UID: \"6e618803-f23d-49c6-896d-f5095313e542\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"
Sep 16 14:08:59.623410 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.623317    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/027136e7-bf36-4d3f-b134-9a0365f1c4da-klusterlet-config\") pod \"klusterlet-addon-workmgr-b674b7c49-bbf4f\" (UID: \"027136e7-bf36-4d3f-b134-9a0365f1c4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.623480 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.623406    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25c2x\" (UniqueName: \"kubernetes.io/projected/6e618803-f23d-49c6-896d-f5095313e542-kube-api-access-25c2x\") pod \"managed-serviceaccount-addon-agent-586c5cdf58-cfb9j\" (UID: \"6e618803-f23d-49c6-896d-f5095313e542\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"
Sep 16 14:08:59.623480 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.623447    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ndn5\" (UniqueName: \"kubernetes.io/projected/027136e7-bf36-4d3f-b134-9a0365f1c4da-kube-api-access-5ndn5\") pod \"klusterlet-addon-workmgr-b674b7c49-bbf4f\" (UID: \"027136e7-bf36-4d3f-b134-9a0365f1c4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.623480 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.623470    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/027136e7-bf36-4d3f-b134-9a0365f1c4da-tmp\") pod \"klusterlet-addon-workmgr-b674b7c49-bbf4f\" (UID: \"027136e7-bf36-4d3f-b134-9a0365f1c4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.648445 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.648419    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-587c9d68f-ljdhz"]
Sep 16 14:08:59.650419 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.650406    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.652916 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.652896    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 14:08:59.653017 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.652970    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 14:08:59.653449 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.653433    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 14:08:59.653526 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.653466    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ffjnt\""
Sep 16 14:08:59.659082 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.659050    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 14:08:59.663897 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.663877    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-587c9d68f-ljdhz"]
Sep 16 14:08:59.724008 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.723942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e618803-f23d-49c6-896d-f5095313e542-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-586c5cdf58-cfb9j\" (UID: \"6e618803-f23d-49c6-896d-f5095313e542\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"
Sep 16 14:08:59.724008 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.723975    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/027136e7-bf36-4d3f-b134-9a0365f1c4da-klusterlet-config\") pod \"klusterlet-addon-workmgr-b674b7c49-bbf4f\" (UID: \"027136e7-bf36-4d3f-b134-9a0365f1c4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.724272 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.724032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25c2x\" (UniqueName: \"kubernetes.io/projected/6e618803-f23d-49c6-896d-f5095313e542-kube-api-access-25c2x\") pod \"managed-serviceaccount-addon-agent-586c5cdf58-cfb9j\" (UID: \"6e618803-f23d-49c6-896d-f5095313e542\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"
Sep 16 14:08:59.724272 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.724071    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ndn5\" (UniqueName: \"kubernetes.io/projected/027136e7-bf36-4d3f-b134-9a0365f1c4da-kube-api-access-5ndn5\") pod \"klusterlet-addon-workmgr-b674b7c49-bbf4f\" (UID: \"027136e7-bf36-4d3f-b134-9a0365f1c4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.724272 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.724101    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vhh9\" (UniqueName: \"kubernetes.io/projected/0af48f71-72de-48c4-ba30-eaff4687fddb-kube-api-access-5vhh9\") pod \"downloads-76f99584f4-g67hh\" (UID: \"0af48f71-72de-48c4-ba30-eaff4687fddb\") " pod="openshift-console/downloads-76f99584f4-g67hh"
Sep 16 14:08:59.724272 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.724133    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/027136e7-bf36-4d3f-b134-9a0365f1c4da-tmp\") pod \"klusterlet-addon-workmgr-b674b7c49-bbf4f\" (UID: \"027136e7-bf36-4d3f-b134-9a0365f1c4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.724545 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.724524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/027136e7-bf36-4d3f-b134-9a0365f1c4da-tmp\") pod \"klusterlet-addon-workmgr-b674b7c49-bbf4f\" (UID: \"027136e7-bf36-4d3f-b134-9a0365f1c4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.726503 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.726482    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e618803-f23d-49c6-896d-f5095313e542-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-586c5cdf58-cfb9j\" (UID: \"6e618803-f23d-49c6-896d-f5095313e542\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"
Sep 16 14:08:59.726613 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.726585    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/027136e7-bf36-4d3f-b134-9a0365f1c4da-klusterlet-config\") pod \"klusterlet-addon-workmgr-b674b7c49-bbf4f\" (UID: \"027136e7-bf36-4d3f-b134-9a0365f1c4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.736009 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.735985    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25c2x\" (UniqueName: \"kubernetes.io/projected/6e618803-f23d-49c6-896d-f5095313e542-kube-api-access-25c2x\") pod \"managed-serviceaccount-addon-agent-586c5cdf58-cfb9j\" (UID: \"6e618803-f23d-49c6-896d-f5095313e542\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"
Sep 16 14:08:59.736009 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.735991    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ndn5\" (UniqueName: \"kubernetes.io/projected/027136e7-bf36-4d3f-b134-9a0365f1c4da-kube-api-access-5ndn5\") pod \"klusterlet-addon-workmgr-b674b7c49-bbf4f\" (UID: \"027136e7-bf36-4d3f-b134-9a0365f1c4da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.783827 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.783795    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:08:59.799902 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.799880    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"
Sep 16 14:08:59.825456 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.825410    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f7b8739-851a-4200-ae5d-0dab8004fd5b-installation-pull-secrets\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.825618 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.825486    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f7b8739-851a-4200-ae5d-0dab8004fd5b-trusted-ca\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.825618 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.825508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f7b8739-851a-4200-ae5d-0dab8004fd5b-image-registry-private-configuration\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.825618 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.825596    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f7b8739-851a-4200-ae5d-0dab8004fd5b-ca-trust-extracted\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.825618 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.825613    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f7b8739-851a-4200-ae5d-0dab8004fd5b-bound-sa-token\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.825863 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.825652    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vhh9\" (UniqueName: \"kubernetes.io/projected/0af48f71-72de-48c4-ba30-eaff4687fddb-kube-api-access-5vhh9\") pod \"downloads-76f99584f4-g67hh\" (UID: \"0af48f71-72de-48c4-ba30-eaff4687fddb\") " pod="openshift-console/downloads-76f99584f4-g67hh"
Sep 16 14:08:59.825863 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.825689    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f7b8739-851a-4200-ae5d-0dab8004fd5b-registry-tls\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.825863 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.825713    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f7b8739-851a-4200-ae5d-0dab8004fd5b-registry-certificates\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.825863 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.825738    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cfn9\" (UniqueName: \"kubernetes.io/projected/4f7b8739-851a-4200-ae5d-0dab8004fd5b-kube-api-access-9cfn9\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.848320 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.848292    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vhh9\" (UniqueName: \"kubernetes.io/projected/0af48f71-72de-48c4-ba30-eaff4687fddb-kube-api-access-5vhh9\") pod \"downloads-76f99584f4-g67hh\" (UID: \"0af48f71-72de-48c4-ba30-eaff4687fddb\") " pod="openshift-console/downloads-76f99584f4-g67hh"
Sep 16 14:08:59.871034 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.871006    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-g67hh"
Sep 16 14:08:59.926662 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.926636    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f7b8739-851a-4200-ae5d-0dab8004fd5b-registry-tls\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.926778 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.926667    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f7b8739-851a-4200-ae5d-0dab8004fd5b-registry-certificates\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.926778 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.926690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cfn9\" (UniqueName: \"kubernetes.io/projected/4f7b8739-851a-4200-ae5d-0dab8004fd5b-kube-api-access-9cfn9\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.926778 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.926737    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f7b8739-851a-4200-ae5d-0dab8004fd5b-installation-pull-secrets\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.926938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.926776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f7b8739-851a-4200-ae5d-0dab8004fd5b-trusted-ca\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.926938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.926803    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f7b8739-851a-4200-ae5d-0dab8004fd5b-image-registry-private-configuration\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.926938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.926855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f7b8739-851a-4200-ae5d-0dab8004fd5b-ca-trust-extracted\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.926938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.926880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f7b8739-851a-4200-ae5d-0dab8004fd5b-bound-sa-token\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.927828 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.927351    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f7b8739-851a-4200-ae5d-0dab8004fd5b-ca-trust-extracted\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.927954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.927891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f7b8739-851a-4200-ae5d-0dab8004fd5b-registry-certificates\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.928047 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.928024    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f7b8739-851a-4200-ae5d-0dab8004fd5b-trusted-ca\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.929482 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.929442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f7b8739-851a-4200-ae5d-0dab8004fd5b-image-registry-private-configuration\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.929482 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.929462    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f7b8739-851a-4200-ae5d-0dab8004fd5b-installation-pull-secrets\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.929649 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.929633    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f7b8739-851a-4200-ae5d-0dab8004fd5b-registry-tls\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.937742 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.937724    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f7b8739-851a-4200-ae5d-0dab8004fd5b-bound-sa-token\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.938006 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.937990    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cfn9\" (UniqueName: \"kubernetes.io/projected/4f7b8739-851a-4200-ae5d-0dab8004fd5b-kube-api-access-9cfn9\") pod \"image-registry-587c9d68f-ljdhz\" (UID: \"4f7b8739-851a-4200-ae5d-0dab8004fd5b\") " pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:08:59.961101 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:08:59.961074    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:09:00.078878 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:00.078839    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"]
Sep 16 14:09:00.083009 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:00.082972    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027136e7_bf36_4d3f_b134_9a0365f1c4da.slice/crio-b5f4b55ddcd9bae00397838d70c4d86ee65367f0e4300a6e55dee98c703dad43 WatchSource:0}: Error finding container b5f4b55ddcd9bae00397838d70c4d86ee65367f0e4300a6e55dee98c703dad43: Status 404 returned error can't find the container with id b5f4b55ddcd9bae00397838d70c4d86ee65367f0e4300a6e55dee98c703dad43
Sep 16 14:09:00.107514 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:00.107489    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j"]
Sep 16 14:09:00.113608 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:00.113559    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e618803_f23d_49c6_896d_f5095313e542.slice/crio-b2dd05b35f4d48f4d639d7f87e5d7f1dd17c6e9f86d8be24c48257e538ccd00e WatchSource:0}: Error finding container b2dd05b35f4d48f4d639d7f87e5d7f1dd17c6e9f86d8be24c48257e538ccd00e: Status 404 returned error can't find the container with id b2dd05b35f4d48f4d639d7f87e5d7f1dd17c6e9f86d8be24c48257e538ccd00e
Sep 16 14:09:00.140027 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:00.140008    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-g67hh"]
Sep 16 14:09:00.141960 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:00.141940    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af48f71_72de_48c4_ba30_eaff4687fddb.slice/crio-3c6701786e71a4130b91e7ff200646f37cc3ae80bcf1082ddcd91acf9d427ed6 WatchSource:0}: Error finding container 3c6701786e71a4130b91e7ff200646f37cc3ae80bcf1082ddcd91acf9d427ed6: Status 404 returned error can't find the container with id 3c6701786e71a4130b91e7ff200646f37cc3ae80bcf1082ddcd91acf9d427ed6
Sep 16 14:09:00.233552 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:00.233530    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-587c9d68f-ljdhz"]
Sep 16 14:09:00.235861 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:00.235837    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7b8739_851a_4200_ae5d_0dab8004fd5b.slice/crio-92d6c11bb8d7229c7852af35dbad017549812c6a75b7e19451318c1228963b72 WatchSource:0}: Error finding container 92d6c11bb8d7229c7852af35dbad017549812c6a75b7e19451318c1228963b72: Status 404 returned error can't find the container with id 92d6c11bb8d7229c7852af35dbad017549812c6a75b7e19451318c1228963b72
Sep 16 14:09:00.252957 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:00.252932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-587c9d68f-ljdhz" event={"ID":"4f7b8739-851a-4200-ae5d-0dab8004fd5b","Type":"ContainerStarted","Data":"92d6c11bb8d7229c7852af35dbad017549812c6a75b7e19451318c1228963b72"}
Sep 16 14:09:00.254044 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:00.254026    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f" event={"ID":"027136e7-bf36-4d3f-b134-9a0365f1c4da","Type":"ContainerStarted","Data":"b5f4b55ddcd9bae00397838d70c4d86ee65367f0e4300a6e55dee98c703dad43"}
Sep 16 14:09:00.255215 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:00.255197    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-g67hh" event={"ID":"0af48f71-72de-48c4-ba30-eaff4687fddb","Type":"ContainerStarted","Data":"3c6701786e71a4130b91e7ff200646f37cc3ae80bcf1082ddcd91acf9d427ed6"}
Sep 16 14:09:00.256264 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:00.256246    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j" event={"ID":"6e618803-f23d-49c6-896d-f5095313e542","Type":"ContainerStarted","Data":"b2dd05b35f4d48f4d639d7f87e5d7f1dd17c6e9f86d8be24c48257e538ccd00e"}
Sep 16 14:09:01.262841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:01.262799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-587c9d68f-ljdhz" event={"ID":"4f7b8739-851a-4200-ae5d-0dab8004fd5b","Type":"ContainerStarted","Data":"dc4ebbe36b1aaf37ac3e9ac1fca1e9f2a529060d501723db489d47fd215ce6ca"}
Sep 16 14:09:01.263289 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:01.263023    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:09:01.284615 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:01.284541    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-587c9d68f-ljdhz" podStartSLOduration=2.284527354 podStartE2EDuration="2.284527354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:01.283347417 +0000 UTC m=+60.760579521" watchObservedRunningTime="2026-09-16 14:09:01.284527354 +0000 UTC m=+60.761759419"
Sep 16 14:09:02.169335 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:02.167749    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mb584"
Sep 16 14:09:04.235236 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:04.235200    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9dtll"
Sep 16 14:09:06.686317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.686276    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:09:06.689160 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.689057    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 14:09:06.699640 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.699609    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1593320-c5df-4f1b-bb35-b90738b70602-original-pull-secret\") pod \"global-pull-secret-syncer-97f7n\" (UID: \"c1593320-c5df-4f1b-bb35-b90738b70602\") " pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:09:06.787211 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.787123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:09:06.789632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.789607    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 14:09:06.800646 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.800622    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3059f22-c414-4a85-9a55-47485f8a88b3-metrics-certs\") pod \"network-metrics-daemon-sjjm9\" (UID: \"e3059f22-c414-4a85-9a55-47485f8a88b3\") " pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:09:06.888338 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.888297    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:09:06.891059 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.891031    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 14:09:06.893198 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.893176    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x4gh8\""
Sep 16 14:09:06.898983 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.898963    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97f7n"
Sep 16 14:09:06.901170 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.900984    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 14:09:06.901170 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.900997    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjjm9"
Sep 16 14:09:06.914697 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:06.914638    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js55n\" (UniqueName: \"kubernetes.io/projected/bebea01f-e7af-48bd-b462-a5cc39c387fc-kube-api-access-js55n\") pod \"network-check-target-8v4g2\" (UID: \"bebea01f-e7af-48bd-b462-a5cc39c387fc\") " pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:09:07.202380 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.202354    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sjjm9"]
Sep 16 14:09:07.202452 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.202426    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-97f7n"]
Sep 16 14:09:07.204369 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:07.204345    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3059f22_c414_4a85_9a55_47485f8a88b3.slice/crio-f9af5d3d018a6fe0fa73af97671ef137431937666f60697ce121b600e053d06e WatchSource:0}: Error finding container f9af5d3d018a6fe0fa73af97671ef137431937666f60697ce121b600e053d06e: Status 404 returned error can't find the container with id f9af5d3d018a6fe0fa73af97671ef137431937666f60697ce121b600e053d06e
Sep 16 14:09:07.206673 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:07.206535    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1593320_c5df_4f1b_bb35_b90738b70602.slice/crio-66f34564f90b4ba51b36cf6490f37c59b2ad9c4ea33c9fc656cdb0e80d7ceeb8 WatchSource:0}: Error finding container 66f34564f90b4ba51b36cf6490f37c59b2ad9c4ea33c9fc656cdb0e80d7ceeb8: Status 404 returned error can't find the container with id 66f34564f90b4ba51b36cf6490f37c59b2ad9c4ea33c9fc656cdb0e80d7ceeb8
Sep 16 14:09:07.207817 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.207776    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8mpwg\""
Sep 16 14:09:07.215626 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.215610    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:09:07.290282 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.290204    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sjjm9" event={"ID":"e3059f22-c414-4a85-9a55-47485f8a88b3","Type":"ContainerStarted","Data":"f9af5d3d018a6fe0fa73af97671ef137431937666f60697ce121b600e053d06e"}
Sep 16 14:09:07.295548 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.295517    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-97f7n" event={"ID":"c1593320-c5df-4f1b-bb35-b90738b70602","Type":"ContainerStarted","Data":"66f34564f90b4ba51b36cf6490f37c59b2ad9c4ea33c9fc656cdb0e80d7ceeb8"}
Sep 16 14:09:07.298197 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.298166    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f" event={"ID":"027136e7-bf36-4d3f-b134-9a0365f1c4da","Type":"ContainerStarted","Data":"04124d5ad3185b279eb7987948d5d2194a1c640a1d4d25c240b3397fc6f5fcc0"}
Sep 16 14:09:07.298842 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.298776    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:09:07.303240 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.300702    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f"
Sep 16 14:09:07.305191 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.305169    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j" event={"ID":"6e618803-f23d-49c6-896d-f5095313e542","Type":"ContainerStarted","Data":"fcfce15972025a766e3ae72c7846b5e1062273c4bd6e1c16063f597b48c2fb9a"}
Sep 16 14:09:07.324300 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.324257    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b674b7c49-bbf4f" podStartSLOduration=1.852466097 podStartE2EDuration="8.324239697s" totalImagesPullingTime="6.4717736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:07.323317012 +0000 UTC m=+66.800549110" watchObservedRunningTime="2026-09-16 14:09:07.324239697 +0000 UTC m=+66.801471772"
Sep 16 14:09:07.364077 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.364026    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586c5cdf58-cfb9j" podStartSLOduration=1.985725479 podStartE2EDuration="8.364007293s" totalImagesPullingTime="6.378281814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:07.362770241 +0000 UTC m=+66.840002317" watchObservedRunningTime="2026-09-16 14:09:07.364007293 +0000 UTC m=+66.841239386"
Sep 16 14:09:07.494764 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.494686    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8v4g2"]
Sep 16 14:09:07.498438 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.497990    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:09:07.498438 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:07.498095    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebea01f_e7af_48bd_b462_a5cc39c387fc.slice/crio-7012e8b8a5227bec81d2a060a56c329a1ab262d9ee1f6ffdca576962f6575c6c WatchSource:0}: Error finding container 7012e8b8a5227bec81d2a060a56c329a1ab262d9ee1f6ffdca576962f6575c6c: Status 404 returned error can't find the container with id 7012e8b8a5227bec81d2a060a56c329a1ab262d9ee1f6ffdca576962f6575c6c
Sep 16 14:09:07.501021 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.500994    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/39204b25-1484-48aa-beff-c86bc7e67b1c-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xzgq8\" (UID: \"39204b25-1484-48aa-beff-c86bc7e67b1c\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:09:07.745976 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.745892    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dkwr7\""
Sep 16 14:09:07.754177 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:07.754151    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"
Sep 16 14:09:08.044945 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:08.044650    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8"]
Sep 16 14:09:08.048043 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:08.048014    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39204b25_1484_48aa_beff_c86bc7e67b1c.slice/crio-d70c85b511746f9569a8d387682192f0c78df30647947def3577334583c864f3 WatchSource:0}: Error finding container d70c85b511746f9569a8d387682192f0c78df30647947def3577334583c864f3: Status 404 returned error can't find the container with id d70c85b511746f9569a8d387682192f0c78df30647947def3577334583c864f3
Sep 16 14:09:08.311228 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:08.311145    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8v4g2" event={"ID":"bebea01f-e7af-48bd-b462-a5cc39c387fc","Type":"ContainerStarted","Data":"7012e8b8a5227bec81d2a060a56c329a1ab262d9ee1f6ffdca576962f6575c6c"}
Sep 16 14:09:08.314210 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:08.314148    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8" event={"ID":"39204b25-1484-48aa-beff-c86bc7e67b1c","Type":"ContainerStarted","Data":"d70c85b511746f9569a8d387682192f0c78df30647947def3577334583c864f3"}
Sep 16 14:09:09.322536 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:09.322497    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sjjm9" event={"ID":"e3059f22-c414-4a85-9a55-47485f8a88b3","Type":"ContainerStarted","Data":"8b9d1562e27cbe7676bb8c21069d4d51a5b798272c633f9fe3cc51adf2251725"}
Sep 16 14:09:09.322998 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:09.322584    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sjjm9" event={"ID":"e3059f22-c414-4a85-9a55-47485f8a88b3","Type":"ContainerStarted","Data":"0663a5131859ff67d4399a92ab7f2313682a628c3028fed452d1a767a0d281c7"}
Sep 16 14:09:09.344984 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:09.344800    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sjjm9" podStartSLOduration=66.08876655 podStartE2EDuration="1m7.344784837s" totalImagesPullingTime="1.256018287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:09.341909322 +0000 UTC m=+68.819141415" watchObservedRunningTime="2026-09-16 14:09:09.344784837 +0000 UTC m=+68.822016912"
Sep 16 14:09:13.217421 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.217378    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"]
Sep 16 14:09:13.243340 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.243307    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"]
Sep 16 14:09:13.243510 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.243490    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.247441 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.245970    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-cmwkr\""
Sep 16 14:09:13.247441 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.246288    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 16 14:09:13.247441 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.246606    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 14:09:13.247441 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.246881    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 14:09:13.247441 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.247063    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 16 14:09:13.247441 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.247290    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 14:09:13.260720 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.259326    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"]
Sep 16 14:09:13.260720 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.259380    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ddkzc"]
Sep 16 14:09:13.260720 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.260126    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.263708 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.263688    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 14:09:13.264525 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.264394    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-6kqc2\""
Sep 16 14:09:13.264525 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.264465    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 14:09:13.264780 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.264764    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 14:09:13.279171 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.279101    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"]
Sep 16 14:09:13.279268 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.279210    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.281302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.281276    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 14:09:13.281414 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.281310    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 14:09:13.281543 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.281528    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g65fn\""
Sep 16 14:09:13.281725 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.281708    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 14:09:13.349999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.349964    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.349999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350011    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-wtmp\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.350252 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350050    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.350252 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350093    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-sys\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.350252 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-textfile\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.350252 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350159    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-tls\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.350252 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350180    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.350252 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350209    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.350252 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsrfc\" (UniqueName: \"kubernetes.io/projected/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-kube-api-access-rsrfc\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-root\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350289    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-accelerators-collector-config\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350354    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-metrics-client-ca\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350381    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8pzt\" (UniqueName: \"kubernetes.io/projected/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-api-access-n8pzt\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/275a6078-f9ea-4055-8683-aa9c5f5355c2-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350441    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/275a6078-f9ea-4055-8683-aa9c5f5355c2-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350466    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqkc7\" (UniqueName: \"kubernetes.io/projected/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-kube-api-access-bqkc7\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350515    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.350690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.350552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451077    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-wtmp\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451155    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-sys\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451181    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-textfile\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451200    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-tls\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451219    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451267    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsrfc\" (UniqueName: \"kubernetes.io/projected/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-kube-api-access-rsrfc\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451280    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-wtmp\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451296    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-root\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451332    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451366    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-accelerators-collector-config\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451385    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-sys\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451402    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-metrics-client-ca\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451428    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8pzt\" (UniqueName: \"kubernetes.io/projected/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-api-access-n8pzt\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.452096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451464    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/275a6078-f9ea-4055-8683-aa9c5f5355c2-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/275a6078-f9ea-4055-8683-aa9c5f5355c2-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqkc7\" (UniqueName: \"kubernetes.io/projected/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-kube-api-access-bqkc7\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451553    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451612    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451334    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-root\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:09:13.451746    2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.451753    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-textfile\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:09:13.451763    2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:09:13.451821    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-tls podName:e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0 nodeName:}" failed. No retries permitted until 2026-09-16 14:09:13.951800024 +0000 UTC m=+73.429032083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-tls") pod "node-exporter-ddkzc" (UID: "e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0") : secret "node-exporter-tls" not found
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:09:13.451838    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-tls podName:275a6078-f9ea-4055-8683-aa9c5f5355c2 nodeName:}" failed. No retries permitted until 2026-09-16 14:09:13.95182906 +0000 UTC m=+73.429061120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-mr884" (UID: "275a6078-f9ea-4055-8683-aa9c5f5355c2") : secret "kube-state-metrics-tls" not found
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.452177    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-metrics-client-ca\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.452750    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/275a6078-f9ea-4055-8683-aa9c5f5355c2-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.453040    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.453206    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-accelerators-collector-config\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.454015 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:09:13.453305    2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 16 14:09:13.454840 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.453358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.454840 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:09:13.453379    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-openshift-state-metrics-tls podName:1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55 nodeName:}" failed. No retries permitted until 2026-09-16 14:09:13.953361946 +0000 UTC m=+73.430594012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-dpdhb" (UID: "1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55") : secret "openshift-state-metrics-tls" not found
Sep 16 14:09:13.454840 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.453359    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/275a6078-f9ea-4055-8683-aa9c5f5355c2-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.454999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.454980    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.462238 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.462178    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.464662 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.464637    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8pzt\" (UniqueName: \"kubernetes.io/projected/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-api-access-n8pzt\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.465171 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.465065    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsrfc\" (UniqueName: \"kubernetes.io/projected/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-kube-api-access-rsrfc\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.465171 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.465099    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.465443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.465424    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqkc7\" (UniqueName: \"kubernetes.io/projected/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-kube-api-access-bqkc7\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.957256 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.957224    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.957449 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.957274    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:13.957449 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.957306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-tls\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.960457 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.960427    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0-node-exporter-tls\") pod \"node-exporter-ddkzc\" (UID: \"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0\") " pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:13.960718 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.960695    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-dpdhb\" (UID: \"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:13.964479 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:13.964450    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/275a6078-f9ea-4055-8683-aa9c5f5355c2-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-mr884\" (UID: \"275a6078-f9ea-4055-8683-aa9c5f5355c2\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:14.160496 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:14.160262    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"
Sep 16 14:09:14.170947 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:14.170922    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"
Sep 16 14:09:14.190515 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:14.190492    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ddkzc"
Sep 16 14:09:21.049490 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:21.049384    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode91ac0d8_4e1b_481f_aad1_8dd7609e3ac0.slice/crio-9dab818f1d24795499f47d2ccc02d8ac2eb325a57da3304d3ef9ff69b8d9f3b9 WatchSource:0}: Error finding container 9dab818f1d24795499f47d2ccc02d8ac2eb325a57da3304d3ef9ff69b8d9f3b9: Status 404 returned error can't find the container with id 9dab818f1d24795499f47d2ccc02d8ac2eb325a57da3304d3ef9ff69b8d9f3b9
Sep 16 14:09:21.311010 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.310971    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-mr884"]
Sep 16 14:09:21.338048 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.338021    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb"]
Sep 16 14:09:21.342141 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:09:21.342111    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef5cfb7_33fd_4a9c_8a5c_7a4f407c9f55.slice/crio-e4d97e3ab12dae8629260cd04121515079a984905ca9cb7b0dad4ca9fca87eea WatchSource:0}: Error finding container e4d97e3ab12dae8629260cd04121515079a984905ca9cb7b0dad4ca9fca87eea: Status 404 returned error can't find the container with id e4d97e3ab12dae8629260cd04121515079a984905ca9cb7b0dad4ca9fca87eea
Sep 16 14:09:21.374960 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.374901    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-g67hh" event={"ID":"0af48f71-72de-48c4-ba30-eaff4687fddb","Type":"ContainerStarted","Data":"85329850f4b0bb43ee7863282e0bb0899f9005333c182d334fdb408acd9d5a4c"}
Sep 16 14:09:21.375313 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.375301    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-g67hh"
Sep 16 14:09:21.376934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.376913    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8v4g2" event={"ID":"bebea01f-e7af-48bd-b462-a5cc39c387fc","Type":"ContainerStarted","Data":"8c48e24e9b4ecbe52333e939a7ba335692902e7e8b780abdad6d0e584ffda7d7"}
Sep 16 14:09:21.377098 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.377084    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:09:21.378884 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.378701    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb" event={"ID":"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55","Type":"ContainerStarted","Data":"e4d97e3ab12dae8629260cd04121515079a984905ca9cb7b0dad4ca9fca87eea"}
Sep 16 14:09:21.380417 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.380360    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddkzc" event={"ID":"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0","Type":"ContainerStarted","Data":"9dab818f1d24795499f47d2ccc02d8ac2eb325a57da3304d3ef9ff69b8d9f3b9"}
Sep 16 14:09:21.381788 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.381726    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884" event={"ID":"275a6078-f9ea-4055-8683-aa9c5f5355c2","Type":"ContainerStarted","Data":"cec72127e32a5ae206509dbafcec4346b49f7afd6f6f07f2addd811b7ea9263f"}
Sep 16 14:09:21.383789 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.383409    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-97f7n" event={"ID":"c1593320-c5df-4f1b-bb35-b90738b70602","Type":"ContainerStarted","Data":"947f40b5c8f19e8473ab03629c7391932ffdb9cab8144292f0616ccfba6d38db"}
Sep 16 14:09:21.384613 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.384600    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-g67hh"
Sep 16 14:09:21.385377 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.385358    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8" event={"ID":"39204b25-1484-48aa-beff-c86bc7e67b1c","Type":"ContainerStarted","Data":"9045ad63180be33695e0da7256e09788c4c45e73767bc92ce197e80f1954fabe"}
Sep 16 14:09:21.396114 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.396060    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-g67hh" podStartSLOduration=1.442443957 podStartE2EDuration="22.396041183s" totalImagesPullingTime="20.953597226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:21.394932375 +0000 UTC m=+80.872164471" watchObservedRunningTime="2026-09-16 14:09:21.396041183 +0000 UTC m=+80.873273260"
Sep 16 14:09:21.417598 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.416754    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8v4g2" podStartSLOduration=65.886259266 podStartE2EDuration="1m19.416737364s" totalImagesPullingTime="13.530478098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:21.41532397 +0000 UTC m=+80.892556042" watchObservedRunningTime="2026-09-16 14:09:21.416737364 +0000 UTC m=+80.893969439"
Sep 16 14:09:21.438494 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:21.438193    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xzgq8" podStartSLOduration=33.458527807 podStartE2EDuration="46.438170079s" totalImagesPullingTime="12.979642272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:21.436326624 +0000 UTC m=+80.913558721" watchObservedRunningTime="2026-09-16 14:09:21.438170079 +0000 UTC m=+80.915402155"
Sep 16 14:09:22.273530 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:22.273497    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-587c9d68f-ljdhz"
Sep 16 14:09:22.304797 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:22.304744    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-97f7n" podStartSLOduration=66.482529876 podStartE2EDuration="1m20.304727439s" totalImagesPullingTime="13.822197563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:21.470744075 +0000 UTC m=+80.947976164" watchObservedRunningTime="2026-09-16 14:09:22.304727439 +0000 UTC m=+81.781959505"
Sep 16 14:09:22.392977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:22.392944    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb" event={"ID":"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55","Type":"ContainerStarted","Data":"23e2dfa0d6628433a781ee51ff8de5bfd7567930fd34a6511f2120a148d151bb"}
Sep 16 14:09:22.393163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:22.393006    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb" event={"ID":"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55","Type":"ContainerStarted","Data":"ac98355c22a866c863bad3184456643a4290dcdfb2dc378666be13304a15943d"}
Sep 16 14:09:23.402454 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:23.402413    2566 generic.go:356] "Generic (PLEG): container finished" podID="e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0" containerID="ed9cb8e92970c6d55e694561fb6811a2c2b21706e81ae570172e56aad9712b39" exitCode=0
Sep 16 14:09:23.402925 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:23.402518    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddkzc" event={"ID":"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0","Type":"ContainerDied","Data":"ed9cb8e92970c6d55e694561fb6811a2c2b21706e81ae570172e56aad9712b39"}
Sep 16 14:09:24.410900 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:24.410808    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb" event={"ID":"1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55","Type":"ContainerStarted","Data":"af6570ae781076b8a1fee5f3555c043c51e22d5a85e023d08a508ebc4e139323"}
Sep 16 14:09:24.414528 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:24.414501    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddkzc" event={"ID":"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0","Type":"ContainerStarted","Data":"0bf2ab3bb5f853e4ad45a328988a77113819dfd2c7e635632b2f2980414b0d48"}
Sep 16 14:09:24.414692 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:24.414548    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddkzc" event={"ID":"e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0","Type":"ContainerStarted","Data":"c23a9fe3e16bf40468eeff900f046cbd79640059626494a28f0046045d3378c6"}
Sep 16 14:09:24.431514 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:24.431459    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-dpdhb" podStartSLOduration=9.027708544 podStartE2EDuration="11.431443515s" totalImagesPullingTime="2.403734971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:24.430243916 +0000 UTC m=+83.907476011" watchObservedRunningTime="2026-09-16 14:09:24.431443515 +0000 UTC m=+83.908675590"
Sep 16 14:09:24.453754 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:24.452650    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ddkzc" podStartSLOduration=9.658408304 podStartE2EDuration="11.452635011s" totalImagesPullingTime="1.693843145s" totalInitContainerRuntime="100.383562ms" isStatefulPod=true podCreationTimestamp="2026-09-16 14:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:24.451400551 +0000 UTC m=+83.928632663" watchObservedRunningTime="2026-09-16 14:09:24.452635011 +0000 UTC m=+83.929867089"
Sep 16 14:09:25.421359 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:25.421326    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884" event={"ID":"275a6078-f9ea-4055-8683-aa9c5f5355c2","Type":"ContainerStarted","Data":"2148de559e216bb5b785dabccc1aedf123ecd16be1b00c230f4b5f0d6f0fdf40"}
Sep 16 14:09:25.421803 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:25.421397    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884" event={"ID":"275a6078-f9ea-4055-8683-aa9c5f5355c2","Type":"ContainerStarted","Data":"0abfac9f2ab676e2273b77522aacd32aa1438c15bbcf4803ad8a529bbdd5196a"}
Sep 16 14:09:26.431396 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:26.431356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884" event={"ID":"275a6078-f9ea-4055-8683-aa9c5f5355c2","Type":"ContainerStarted","Data":"e38a0774edcef1edc645a171f7ddd8f51aeede308cdf19dc6210961d0afeea00"}
Sep 16 14:09:26.512491 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:26.512430    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-mr884" podStartSLOduration=9.743115155 podStartE2EDuration="13.51241135s" totalImagesPullingTime="3.769296195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:26.510734333 +0000 UTC m=+85.987966425" watchObservedRunningTime="2026-09-16 14:09:26.51241135 +0000 UTC m=+85.989643426"
Sep 16 14:09:52.395580 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:09:52.395546    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8v4g2"
Sep 16 14:13:00.879188 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:00.879160    2566 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 14:13:22.203986 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.203944    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"]
Sep 16 14:13:22.205952 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.205935    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"
Sep 16 14:13:22.209976 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.209958    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 14:13:22.210094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.210070    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4hp29\""
Sep 16 14:13:22.210202 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.210187    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 14:13:22.226864 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.226843    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"]
Sep 16 14:13:22.256037 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.256012    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8460a26-6999-4efd-83a4-e4434ee8ec39-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9dlxl\" (UID: \"f8460a26-6999-4efd-83a4-e4434ee8ec39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"
Sep 16 14:13:22.256163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.256064    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75gwg\" (UniqueName: \"kubernetes.io/projected/f8460a26-6999-4efd-83a4-e4434ee8ec39-kube-api-access-75gwg\") pod \"cert-manager-cainjector-559cdd8c87-9dlxl\" (UID: \"f8460a26-6999-4efd-83a4-e4434ee8ec39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"
Sep 16 14:13:22.356963 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.356926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8460a26-6999-4efd-83a4-e4434ee8ec39-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9dlxl\" (UID: \"f8460a26-6999-4efd-83a4-e4434ee8ec39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"
Sep 16 14:13:22.357125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.356973    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75gwg\" (UniqueName: \"kubernetes.io/projected/f8460a26-6999-4efd-83a4-e4434ee8ec39-kube-api-access-75gwg\") pod \"cert-manager-cainjector-559cdd8c87-9dlxl\" (UID: \"f8460a26-6999-4efd-83a4-e4434ee8ec39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"
Sep 16 14:13:22.366674 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.366646    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8460a26-6999-4efd-83a4-e4434ee8ec39-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9dlxl\" (UID: \"f8460a26-6999-4efd-83a4-e4434ee8ec39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"
Sep 16 14:13:22.367486 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.367461    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75gwg\" (UniqueName: \"kubernetes.io/projected/f8460a26-6999-4efd-83a4-e4434ee8ec39-kube-api-access-75gwg\") pod \"cert-manager-cainjector-559cdd8c87-9dlxl\" (UID: \"f8460a26-6999-4efd-83a4-e4434ee8ec39\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"
Sep 16 14:13:22.515042 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.514957    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"
Sep 16 14:13:22.780971 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.780949    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl"]
Sep 16 14:13:22.783420 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:13:22.783394    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8460a26_6999_4efd_83a4_e4434ee8ec39.slice/crio-8e25e253fd8b9312187d31d456ff212460f9141c53d707dacd3c512368539606 WatchSource:0}: Error finding container 8e25e253fd8b9312187d31d456ff212460f9141c53d707dacd3c512368539606: Status 404 returned error can't find the container with id 8e25e253fd8b9312187d31d456ff212460f9141c53d707dacd3c512368539606
Sep 16 14:13:22.785402 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:22.785381    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:13:23.277340 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:23.277301    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl" event={"ID":"f8460a26-6999-4efd-83a4-e4434ee8ec39","Type":"ContainerStarted","Data":"8e25e253fd8b9312187d31d456ff212460f9141c53d707dacd3c512368539606"}
Sep 16 14:13:27.295781 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:27.295744    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl" event={"ID":"f8460a26-6999-4efd-83a4-e4434ee8ec39","Type":"ContainerStarted","Data":"19cebd37594ef223b7495cde44ab223d86a79a1dcdb7c129d4b249a6b3d00699"}
Sep 16 14:13:27.329357 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:27.329307    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9dlxl" podStartSLOduration=1.520538392 podStartE2EDuration="5.329291796s" totalImagesPullingTime="3.808753404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:13:27.328447669 +0000 UTC m=+326.805679746" watchObservedRunningTime="2026-09-16 14:13:27.329291796 +0000 UTC m=+326.806523871"
Sep 16 14:13:47.812538 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.812503    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"]
Sep 16 14:13:47.817004 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.816987    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.822275 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.822247    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 14:13:47.822653 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.822634    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 14:13:47.822751 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.822654    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 14:13:47.822887 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.822871    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:13:47.823658 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.823641    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ll6mg\""
Sep 16 14:13:47.825034 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.825017    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 14:13:47.838463 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.838440    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"]
Sep 16 14:13:47.841288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.841265    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0da4d074-3940-4073-b4d0-f85d1ddda6f8-manager-config\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.841384 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.841308    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng7tx\" (UniqueName: \"kubernetes.io/projected/0da4d074-3940-4073-b4d0-f85d1ddda6f8-kube-api-access-ng7tx\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.841384 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.841347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da4d074-3940-4073-b4d0-f85d1ddda6f8-cert\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.841466 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.841393    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0da4d074-3940-4073-b4d0-f85d1ddda6f8-metrics-cert\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.942525 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.942493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0da4d074-3940-4073-b4d0-f85d1ddda6f8-metrics-cert\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.942718 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.942537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0da4d074-3940-4073-b4d0-f85d1ddda6f8-manager-config\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.942718 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.942559    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng7tx\" (UniqueName: \"kubernetes.io/projected/0da4d074-3940-4073-b4d0-f85d1ddda6f8-kube-api-access-ng7tx\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.942718 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.942606    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da4d074-3940-4073-b4d0-f85d1ddda6f8-cert\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.943257 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.943229    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0da4d074-3940-4073-b4d0-f85d1ddda6f8-manager-config\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.945234 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.945204    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0da4d074-3940-4073-b4d0-f85d1ddda6f8-metrics-cert\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.945338 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.945302    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da4d074-3940-4073-b4d0-f85d1ddda6f8-cert\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:47.951715 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:47.951691    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng7tx\" (UniqueName: \"kubernetes.io/projected/0da4d074-3940-4073-b4d0-f85d1ddda6f8-kube-api-access-ng7tx\") pod \"lws-controller-manager-8544d9d757-jkwgv\" (UID: \"0da4d074-3940-4073-b4d0-f85d1ddda6f8\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:48.125984 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:48.125892    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:48.402802 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:48.402630    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"]
Sep 16 14:13:48.405484 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:13:48.405459    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da4d074_3940_4073_b4d0_f85d1ddda6f8.slice/crio-47aa53e6153549a9297edcbf5632f717cfd53544f64c64c81fa50d63f3c6a8cf WatchSource:0}: Error finding container 47aa53e6153549a9297edcbf5632f717cfd53544f64c64c81fa50d63f3c6a8cf: Status 404 returned error can't find the container with id 47aa53e6153549a9297edcbf5632f717cfd53544f64c64c81fa50d63f3c6a8cf
Sep 16 14:13:49.379732 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:49.379692    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv" event={"ID":"0da4d074-3940-4073-b4d0-f85d1ddda6f8","Type":"ContainerStarted","Data":"47aa53e6153549a9297edcbf5632f717cfd53544f64c64c81fa50d63f3c6a8cf"}
Sep 16 14:13:51.388363 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:51.388275    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv" event={"ID":"0da4d074-3940-4073-b4d0-f85d1ddda6f8","Type":"ContainerStarted","Data":"65ad12488bd9ee5d277187604d219e0644c4fffdf30e0380fe997d11506eded8"}
Sep 16 14:13:51.388777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:51.388394    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:13:51.406837 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:51.406792    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv" podStartSLOduration=1.732404515 podStartE2EDuration="4.406778272s" totalImagesPullingTime="2.674373757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:13:51.405979153 +0000 UTC m=+350.883211239" watchObservedRunningTime="2026-09-16 14:13:51.406778272 +0000 UTC m=+350.884010407"
Sep 16 14:13:54.287086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.287001    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"]
Sep 16 14:13:54.290374 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.290357    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.295850 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.295829    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 14:13:54.295850 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.295841    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 14:13:54.296096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.296081    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-plblj\""
Sep 16 14:13:54.297296 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.297269    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 14:13:54.300766 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.300751    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 14:13:54.327555 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.327531    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"]
Sep 16 14:13:54.393370 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.393336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn4sf\" (UniqueName: \"kubernetes.io/projected/362e4670-8270-46a5-9d16-843ec6470a47-kube-api-access-qn4sf\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.393520 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.393374    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/362e4670-8270-46a5-9d16-843ec6470a47-manifests\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.393520 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.393394    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/362e4670-8270-46a5-9d16-843ec6470a47-tmp\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.393520 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.393453    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/362e4670-8270-46a5-9d16-843ec6470a47-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.393520 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.393473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/362e4670-8270-46a5-9d16-843ec6470a47-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.493982 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.493948    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/362e4670-8270-46a5-9d16-843ec6470a47-manifests\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.493982 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.493989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/362e4670-8270-46a5-9d16-843ec6470a47-tmp\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.494225 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.494044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/362e4670-8270-46a5-9d16-843ec6470a47-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.494225 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.494072    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/362e4670-8270-46a5-9d16-843ec6470a47-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.494225 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.494115    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn4sf\" (UniqueName: \"kubernetes.io/projected/362e4670-8270-46a5-9d16-843ec6470a47-kube-api-access-qn4sf\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.494427 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.494398    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/362e4670-8270-46a5-9d16-843ec6470a47-manifests\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.496620 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.496597    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/362e4670-8270-46a5-9d16-843ec6470a47-tmp\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.496843 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.496822    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/362e4670-8270-46a5-9d16-843ec6470a47-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.496937 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.496923    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/362e4670-8270-46a5-9d16-843ec6470a47-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.503479 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.503455    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn4sf\" (UniqueName: \"kubernetes.io/projected/362e4670-8270-46a5-9d16-843ec6470a47-kube-api-access-qn4sf\") pod \"opendatahub-operator-controller-manager-58d8b99777-j6bg5\" (UID: \"362e4670-8270-46a5-9d16-843ec6470a47\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.600102 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.600023    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:13:54.877803 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:54.877720    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"]
Sep 16 14:13:54.888292 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:13:54.888257    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod362e4670_8270_46a5_9d16_843ec6470a47.slice/crio-7356f47b9367b0e93cf7f5ec72887df7738639718d5ae2a817d2f97670cb995d WatchSource:0}: Error finding container 7356f47b9367b0e93cf7f5ec72887df7738639718d5ae2a817d2f97670cb995d: Status 404 returned error can't find the container with id 7356f47b9367b0e93cf7f5ec72887df7738639718d5ae2a817d2f97670cb995d
Sep 16 14:13:55.406331 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:55.406296    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5" event={"ID":"362e4670-8270-46a5-9d16-843ec6470a47","Type":"ContainerStarted","Data":"7356f47b9367b0e93cf7f5ec72887df7738639718d5ae2a817d2f97670cb995d"}
Sep 16 14:13:59.425559 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:13:59.425521    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5" event={"ID":"362e4670-8270-46a5-9d16-843ec6470a47","Type":"ContainerStarted","Data":"425a90e243500b7b79bc4bb4f0bceb810e520ab75430d7fda0932e254444255b"}
Sep 16 14:14:00.430974 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:00.430891    2566 generic.go:356] "Generic (PLEG): container finished" podID="362e4670-8270-46a5-9d16-843ec6470a47" containerID="425a90e243500b7b79bc4bb4f0bceb810e520ab75430d7fda0932e254444255b" exitCode=0
Sep 16 14:14:00.430974 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:00.430936    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5" event={"ID":"362e4670-8270-46a5-9d16-843ec6470a47","Type":"ContainerDied","Data":"425a90e243500b7b79bc4bb4f0bceb810e520ab75430d7fda0932e254444255b"}
Sep 16 14:14:01.437217 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:01.437179    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5" event={"ID":"362e4670-8270-46a5-9d16-843ec6470a47","Type":"ContainerStarted","Data":"23fe2725c41ccc3d4d89abed4f176f980637f79d4e1feea55e19374ff47a52f5"}
Sep 16 14:14:01.437672 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:01.437313    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:14:01.467106 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:01.467044    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5" podStartSLOduration=2.020603252 podStartE2EDuration="7.467027101s" totalImagesPullingTime="4.420432585s" totalInitContainerRuntime="1.025991264s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:01.466445843 +0000 UTC m=+360.943677966" watchObservedRunningTime="2026-09-16 14:14:01.467027101 +0000 UTC m=+360.944259175"
Sep 16 14:14:02.395365 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:02.395338    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-jkwgv"
Sep 16 14:14:12.444241 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:12.444210    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-j6bg5"
Sep 16 14:14:18.794847 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.794817    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"]
Sep 16 14:14:18.805419 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.805396    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:18.812790 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.812767    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hbc8q\""
Sep 16 14:14:18.812913 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.812799    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 14:14:18.812913 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.812767    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 14:14:18.812913 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.812798    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 14:14:18.813428 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.813413    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 14:14:18.822953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.822929    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"]
Sep 16 14:14:18.888782 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.888748    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afa151bc-3aa6-443f-ad03-987f53865732-tmp\") pod \"kube-auth-proxy-57bfccdb6c-srkf4\" (UID: \"afa151bc-3aa6-443f-ad03-987f53865732\") " pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:18.888920 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.888790    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnd8p\" (UniqueName: \"kubernetes.io/projected/afa151bc-3aa6-443f-ad03-987f53865732-kube-api-access-tnd8p\") pod \"kube-auth-proxy-57bfccdb6c-srkf4\" (UID: \"afa151bc-3aa6-443f-ad03-987f53865732\") " pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:18.888920 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.888811    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afa151bc-3aa6-443f-ad03-987f53865732-tls-certs\") pod \"kube-auth-proxy-57bfccdb6c-srkf4\" (UID: \"afa151bc-3aa6-443f-ad03-987f53865732\") " pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:18.989219 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.989196    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afa151bc-3aa6-443f-ad03-987f53865732-tmp\") pod \"kube-auth-proxy-57bfccdb6c-srkf4\" (UID: \"afa151bc-3aa6-443f-ad03-987f53865732\") " pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:18.989346 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.989231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnd8p\" (UniqueName: \"kubernetes.io/projected/afa151bc-3aa6-443f-ad03-987f53865732-kube-api-access-tnd8p\") pod \"kube-auth-proxy-57bfccdb6c-srkf4\" (UID: \"afa151bc-3aa6-443f-ad03-987f53865732\") " pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:18.989346 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.989253    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afa151bc-3aa6-443f-ad03-987f53865732-tls-certs\") pod \"kube-auth-proxy-57bfccdb6c-srkf4\" (UID: \"afa151bc-3aa6-443f-ad03-987f53865732\") " pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:18.991706 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.991680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afa151bc-3aa6-443f-ad03-987f53865732-tmp\") pod \"kube-auth-proxy-57bfccdb6c-srkf4\" (UID: \"afa151bc-3aa6-443f-ad03-987f53865732\") " pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:18.991929 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:18.991908    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afa151bc-3aa6-443f-ad03-987f53865732-tls-certs\") pod \"kube-auth-proxy-57bfccdb6c-srkf4\" (UID: \"afa151bc-3aa6-443f-ad03-987f53865732\") " pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:19.011470 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:19.011442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnd8p\" (UniqueName: \"kubernetes.io/projected/afa151bc-3aa6-443f-ad03-987f53865732-kube-api-access-tnd8p\") pod \"kube-auth-proxy-57bfccdb6c-srkf4\" (UID: \"afa151bc-3aa6-443f-ad03-987f53865732\") " pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:19.114508 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:19.114413    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"
Sep 16 14:14:19.397932 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:19.397856    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4"]
Sep 16 14:14:19.402520 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:14:19.402485    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa151bc_3aa6_443f_ad03_987f53865732.slice/crio-218637817f9d7aac0369abceba147a6d89e1c6b42607beb741af4173fdf119e2 WatchSource:0}: Error finding container 218637817f9d7aac0369abceba147a6d89e1c6b42607beb741af4173fdf119e2: Status 404 returned error can't find the container with id 218637817f9d7aac0369abceba147a6d89e1c6b42607beb741af4173fdf119e2
Sep 16 14:14:19.516418 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:19.516381    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4" event={"ID":"afa151bc-3aa6-443f-ad03-987f53865732","Type":"ContainerStarted","Data":"218637817f9d7aac0369abceba147a6d89e1c6b42607beb741af4173fdf119e2"}
Sep 16 14:14:22.532073 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:22.532037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4" event={"ID":"afa151bc-3aa6-443f-ad03-987f53865732","Type":"ContainerStarted","Data":"9f48a5daed1034a81de2fe34460557c08ae19a720aae04e2fea6c59d532d2eee"}
Sep 16 14:14:22.552796 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:14:22.552744    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57bfccdb6c-srkf4" podStartSLOduration=1.842073404 podStartE2EDuration="4.552727942s" totalImagesPullingTime="2.710654538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:22.550836565 +0000 UTC m=+382.028068640" watchObservedRunningTime="2026-09-16 14:14:22.552727942 +0000 UTC m=+382.029960076"
Sep 16 14:16:05.572352 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.572321    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"]
Sep 16 14:16:05.575340 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.575320    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:05.579450 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.579428    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 14:16:05.579450 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.579438    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 16 14:16:05.580074 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.580055    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 16 14:16:05.580152 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.580065    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4hfdv\""
Sep 16 14:16:05.581434 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.581414    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 14:16:05.611549 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.611519    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/90a13e81-e37b-457c-aa36-4594f664bd6a-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:05.611549 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.611552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/90a13e81-e37b-457c-aa36-4594f664bd6a-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:05.611777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.611625    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9wd7\" (UniqueName: \"kubernetes.io/projected/90a13e81-e37b-457c-aa36-4594f664bd6a-kube-api-access-q9wd7\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:05.630270 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.630235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"]
Sep 16 14:16:05.712163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.712127    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/90a13e81-e37b-457c-aa36-4594f664bd6a-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:05.712163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.712162    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/90a13e81-e37b-457c-aa36-4594f664bd6a-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:05.712413 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.712194    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9wd7\" (UniqueName: \"kubernetes.io/projected/90a13e81-e37b-457c-aa36-4594f664bd6a-kube-api-access-q9wd7\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:05.712413 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:16:05.712288    2566 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 16 14:16:05.712413 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:16:05.712362    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90a13e81-e37b-457c-aa36-4594f664bd6a-plugin-serving-cert podName:90a13e81-e37b-457c-aa36-4594f664bd6a nodeName:}" failed. No retries permitted until 2026-09-16 14:16:06.212340487 +0000 UTC m=+485.689572539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/90a13e81-e37b-457c-aa36-4594f664bd6a-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-9rbqq" (UID: "90a13e81-e37b-457c-aa36-4594f664bd6a") : secret "plugin-serving-cert" not found
Sep 16 14:16:05.712816 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.712796    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/90a13e81-e37b-457c-aa36-4594f664bd6a-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:05.728197 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:05.728174    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9wd7\" (UniqueName: \"kubernetes.io/projected/90a13e81-e37b-457c-aa36-4594f664bd6a-kube-api-access-q9wd7\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:06.216440 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:06.216405    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/90a13e81-e37b-457c-aa36-4594f664bd6a-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:06.218992 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:06.218967    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/90a13e81-e37b-457c-aa36-4594f664bd6a-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9rbqq\" (UID: \"90a13e81-e37b-457c-aa36-4594f664bd6a\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:06.485379 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:06.485297    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"
Sep 16 14:16:06.758025 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:06.757906    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq"]
Sep 16 14:16:06.760823 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:16:06.760796    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90a13e81_e37b_457c_aa36_4594f664bd6a.slice/crio-a02a5fe2c550157b32c4af81e4f332344a319584d737ae4fb439ae330b6e4c72 WatchSource:0}: Error finding container a02a5fe2c550157b32c4af81e4f332344a319584d737ae4fb439ae330b6e4c72: Status 404 returned error can't find the container with id a02a5fe2c550157b32c4af81e4f332344a319584d737ae4fb439ae330b6e4c72
Sep 16 14:16:06.947116 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:06.947073    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq" event={"ID":"90a13e81-e37b-457c-aa36-4594f664bd6a","Type":"ContainerStarted","Data":"a02a5fe2c550157b32c4af81e4f332344a319584d737ae4fb439ae330b6e4c72"}
Sep 16 14:16:11.973922 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:11.973886    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq" event={"ID":"90a13e81-e37b-457c-aa36-4594f664bd6a","Type":"ContainerStarted","Data":"20d1f5ad8e0ccf5d66db8415beb76369f79b02502698769e18742226af6b4bf8"}
Sep 16 14:16:11.998817 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:16:11.998771    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9rbqq" podStartSLOduration=2.222962566 podStartE2EDuration="6.998751682s" totalImagesPullingTime="4.775789116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:16:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:16:11.998737209 +0000 UTC m=+491.475969286" watchObservedRunningTime="2026-09-16 14:16:11.998751682 +0000 UTC m=+491.475983755"
Sep 16 14:17:01.315117 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.315032    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:17:01.318135 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.318116    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:01.324155 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.324132    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 14:17:01.352991 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.352959    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/010db054-aeb0-48b5-beec-9f94658ffe41-config-file\") pod \"limitador-limitador-84994f7ffc-t2hqt\" (UID: \"010db054-aeb0-48b5-beec-9f94658ffe41\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:01.353137 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.352998    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xscz7\" (UniqueName: \"kubernetes.io/projected/010db054-aeb0-48b5-beec-9f94658ffe41-kube-api-access-xscz7\") pod \"limitador-limitador-84994f7ffc-t2hqt\" (UID: \"010db054-aeb0-48b5-beec-9f94658ffe41\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:01.355682 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.355657    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:17:01.401482 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.401453    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:17:01.454068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.454043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/010db054-aeb0-48b5-beec-9f94658ffe41-config-file\") pod \"limitador-limitador-84994f7ffc-t2hqt\" (UID: \"010db054-aeb0-48b5-beec-9f94658ffe41\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:01.454221 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.454075    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xscz7\" (UniqueName: \"kubernetes.io/projected/010db054-aeb0-48b5-beec-9f94658ffe41-kube-api-access-xscz7\") pod \"limitador-limitador-84994f7ffc-t2hqt\" (UID: \"010db054-aeb0-48b5-beec-9f94658ffe41\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:01.454664 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.454647    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/010db054-aeb0-48b5-beec-9f94658ffe41-config-file\") pod \"limitador-limitador-84994f7ffc-t2hqt\" (UID: \"010db054-aeb0-48b5-beec-9f94658ffe41\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:01.477672 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.477638    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xscz7\" (UniqueName: \"kubernetes.io/projected/010db054-aeb0-48b5-beec-9f94658ffe41-kube-api-access-xscz7\") pod \"limitador-limitador-84994f7ffc-t2hqt\" (UID: \"010db054-aeb0-48b5-beec-9f94658ffe41\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:01.628179 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.628090    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:01.860735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.860701    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-zgbnx"]
Sep 16 14:17:01.863938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.863921    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-zgbnx"
Sep 16 14:17:01.866423 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.866403    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-slcv4\""
Sep 16 14:17:01.871695 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.871675    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zgbnx"]
Sep 16 14:17:01.910299 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.910277    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:17:01.912141 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:17:01.912114    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010db054_aeb0_48b5_beec_9f94658ffe41.slice/crio-853921964c7671cc40fc9423440a3f05786ee7291e535aa3f87d4afdbba1e306 WatchSource:0}: Error finding container 853921964c7671cc40fc9423440a3f05786ee7291e535aa3f87d4afdbba1e306: Status 404 returned error can't find the container with id 853921964c7671cc40fc9423440a3f05786ee7291e535aa3f87d4afdbba1e306
Sep 16 14:17:01.957665 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:01.957615    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvswh\" (UniqueName: \"kubernetes.io/projected/bb77f0d0-0599-499b-a9dd-c35de6684cae-kube-api-access-qvswh\") pod \"authorino-59c974bd48-zgbnx\" (UID: \"bb77f0d0-0599-499b-a9dd-c35de6684cae\") " pod="kuadrant-system/authorino-59c974bd48-zgbnx"
Sep 16 14:17:02.058700 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:02.058666    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvswh\" (UniqueName: \"kubernetes.io/projected/bb77f0d0-0599-499b-a9dd-c35de6684cae-kube-api-access-qvswh\") pod \"authorino-59c974bd48-zgbnx\" (UID: \"bb77f0d0-0599-499b-a9dd-c35de6684cae\") " pod="kuadrant-system/authorino-59c974bd48-zgbnx"
Sep 16 14:17:02.068208 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:02.068184    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvswh\" (UniqueName: \"kubernetes.io/projected/bb77f0d0-0599-499b-a9dd-c35de6684cae-kube-api-access-qvswh\") pod \"authorino-59c974bd48-zgbnx\" (UID: \"bb77f0d0-0599-499b-a9dd-c35de6684cae\") " pod="kuadrant-system/authorino-59c974bd48-zgbnx"
Sep 16 14:17:02.169483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:02.169397    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt" event={"ID":"010db054-aeb0-48b5-beec-9f94658ffe41","Type":"ContainerStarted","Data":"853921964c7671cc40fc9423440a3f05786ee7291e535aa3f87d4afdbba1e306"}
Sep 16 14:17:02.173379 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:02.173343    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-zgbnx"
Sep 16 14:17:02.462440 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:02.462403    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zgbnx"]
Sep 16 14:17:02.465130 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:17:02.465104    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb77f0d0_0599_499b_a9dd_c35de6684cae.slice/crio-911358de8431321e528418ba398bb072dc90b8d893f81c8a46c01b53f2b81c88 WatchSource:0}: Error finding container 911358de8431321e528418ba398bb072dc90b8d893f81c8a46c01b53f2b81c88: Status 404 returned error can't find the container with id 911358de8431321e528418ba398bb072dc90b8d893f81c8a46c01b53f2b81c88
Sep 16 14:17:03.177327 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:03.177286    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-zgbnx" event={"ID":"bb77f0d0-0599-499b-a9dd-c35de6684cae","Type":"ContainerStarted","Data":"911358de8431321e528418ba398bb072dc90b8d893f81c8a46c01b53f2b81c88"}
Sep 16 14:17:05.187898 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:05.187858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt" event={"ID":"010db054-aeb0-48b5-beec-9f94658ffe41","Type":"ContainerStarted","Data":"22a787d756e1c9668bae30efa0924471f57d0be8d41d26f974b0598b3b486c24"}
Sep 16 14:17:05.188326 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:05.187938    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:05.189504 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:05.189484    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-zgbnx" event={"ID":"bb77f0d0-0599-499b-a9dd-c35de6684cae","Type":"ContainerStarted","Data":"da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e"}
Sep 16 14:17:05.204470 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:05.204425    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt" podStartSLOduration=1.438564638 podStartE2EDuration="4.204413893s" totalImagesPullingTime="2.765849255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:17:05.203374562 +0000 UTC m=+544.680606649" watchObservedRunningTime="2026-09-16 14:17:05.204413893 +0000 UTC m=+544.681645967"
Sep 16 14:17:05.217330 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:05.217278    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-zgbnx" podStartSLOduration=1.998467539 podStartE2EDuration="4.217261472s" totalImagesPullingTime="2.218793933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:17:05.216722077 +0000 UTC m=+544.693954153" watchObservedRunningTime="2026-09-16 14:17:05.217261472 +0000 UTC m=+544.694493547"
Sep 16 14:17:16.196531 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:16.196500    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"
Sep 16 14:17:34.786932 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:34.786896    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zgbnx"]
Sep 16 14:17:34.787423 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:34.787065    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-zgbnx" podUID="bb77f0d0-0599-499b-a9dd-c35de6684cae" containerName="authorino" containerID="cri-o://da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e" gracePeriod=30
Sep 16 14:17:35.031504 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.031476    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-zgbnx"
Sep 16 14:17:35.128667 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.128536    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvswh\" (UniqueName: \"kubernetes.io/projected/bb77f0d0-0599-499b-a9dd-c35de6684cae-kube-api-access-qvswh\") pod \"bb77f0d0-0599-499b-a9dd-c35de6684cae\" (UID: \"bb77f0d0-0599-499b-a9dd-c35de6684cae\") "
Sep 16 14:17:35.130863 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.130834    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb77f0d0-0599-499b-a9dd-c35de6684cae-kube-api-access-qvswh" (OuterVolumeSpecName: "kube-api-access-qvswh") pod "bb77f0d0-0599-499b-a9dd-c35de6684cae" (UID: "bb77f0d0-0599-499b-a9dd-c35de6684cae"). InnerVolumeSpecName "kube-api-access-qvswh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:17:35.229384 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.229347    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvswh\" (UniqueName: \"kubernetes.io/projected/bb77f0d0-0599-499b-a9dd-c35de6684cae-kube-api-access-qvswh\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:17:35.311493 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.311454    2566 generic.go:356] "Generic (PLEG): container finished" podID="bb77f0d0-0599-499b-a9dd-c35de6684cae" containerID="da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e" exitCode=0
Sep 16 14:17:35.311666 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.311503    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-zgbnx"
Sep 16 14:17:35.311666 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.311545    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-zgbnx" event={"ID":"bb77f0d0-0599-499b-a9dd-c35de6684cae","Type":"ContainerDied","Data":"da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e"}
Sep 16 14:17:35.311666 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.311599    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-zgbnx" event={"ID":"bb77f0d0-0599-499b-a9dd-c35de6684cae","Type":"ContainerDied","Data":"911358de8431321e528418ba398bb072dc90b8d893f81c8a46c01b53f2b81c88"}
Sep 16 14:17:35.311666 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.311617    2566 scope.go:117] "RemoveContainer" containerID="da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e"
Sep 16 14:17:35.320104 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.320090    2566 scope.go:117] "RemoveContainer" containerID="da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e"
Sep 16 14:17:35.320365 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:17:35.320346    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e\": container with ID starting with da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e not found: ID does not exist" containerID="da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e"
Sep 16 14:17:35.320435 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.320376    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e"} err="failed to get container status \"da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e\": rpc error: code = NotFound desc = could not find container \"da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e\": container with ID starting with da23e41afe23c2281f9f32a954da28c6deacec81e29a2c6d3f15332ac933bf8e not found: ID does not exist"
Sep 16 14:17:35.337139 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.337112    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zgbnx"]
Sep 16 14:17:35.341779 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:35.341760    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zgbnx"]
Sep 16 14:17:36.982771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:17:36.982740    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb77f0d0-0599-499b-a9dd-c35de6684cae" path="/var/lib/kubelet/pods/bb77f0d0-0599-499b-a9dd-c35de6684cae/volumes"
Sep 16 14:18:17.493938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:17.493904    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:18:30.408259 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.408183    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"]
Sep 16 14:18:30.410562 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.408654    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb77f0d0-0599-499b-a9dd-c35de6684cae" containerName="authorino"
Sep 16 14:18:30.410562 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.408670    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb77f0d0-0599-499b-a9dd-c35de6684cae" containerName="authorino"
Sep 16 14:18:30.410562 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.408749    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb77f0d0-0599-499b-a9dd-c35de6684cae" containerName="authorino"
Sep 16 14:18:30.411733 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.411716    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.414621 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.414593    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 16 14:18:30.414756 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.414606    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 14:18:30.414756 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.414692    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vgn9b\""
Sep 16 14:18:30.414756 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.414702    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 14:18:30.424688 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.424665    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"]
Sep 16 14:18:30.469387 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.469352    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.469387 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.469389    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.469628 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.469409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfj5r\" (UniqueName: \"kubernetes.io/projected/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-kube-api-access-xfj5r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.469628 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.469437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.469628 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.469520    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.469628 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.469554    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.570961 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.570929    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.570961 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.570968    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.571184 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.571027    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.571184 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.571055    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.571184 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.571082    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfj5r\" (UniqueName: \"kubernetes.io/projected/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-kube-api-access-xfj5r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.571184 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.571123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.571412 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.571394    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.571456 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.571423    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.571523 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.571453    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.573433 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.573403    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.573677 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.573660    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.580128 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.580106    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfj5r\" (UniqueName: \"kubernetes.io/projected/4651345f-a608-4d1f-9e5e-d4bc1d46d6b3-kube-api-access-xfj5r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv\" (UID: \"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.721810 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.721776    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:30.901984 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.901944    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:18:30.994002 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.993983    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"]
Sep 16 14:18:30.996083 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:18:30.996058    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4651345f_a608_4d1f_9e5e_d4bc1d46d6b3.slice/crio-976b1349d40dffa4b636bb933507840b1c2ed9949025cc3413b7934fac05d544 WatchSource:0}: Error finding container 976b1349d40dffa4b636bb933507840b1c2ed9949025cc3413b7934fac05d544: Status 404 returned error can't find the container with id 976b1349d40dffa4b636bb933507840b1c2ed9949025cc3413b7934fac05d544
Sep 16 14:18:30.997340 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:30.997325    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:18:31.548490 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:31.548455    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv" event={"ID":"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3","Type":"ContainerStarted","Data":"976b1349d40dffa4b636bb933507840b1c2ed9949025cc3413b7934fac05d544"}
Sep 16 14:18:32.554276 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:32.554241    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv" event={"ID":"4651345f-a608-4d1f-9e5e-d4bc1d46d6b3","Type":"ContainerStarted","Data":"c438533165c355c922fdb094ac91e458d8d4c20213cdd90637dc153d237881d0"}
Sep 16 14:18:32.574112 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:32.574052    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv" podStartSLOduration=1.524250534 podStartE2EDuration="2.574036993s" totalImagesPullingTime="1.049786459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:32.573810932 +0000 UTC m=+632.051043017" watchObservedRunningTime="2026-09-16 14:18:32.574036993 +0000 UTC m=+632.051269068"
Sep 16 14:18:40.722133 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:40.722097    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:40.722521 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:40.722182    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:40.734746 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:40.734718    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:41.607457 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:41.607426    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-msjjv"
Sep 16 14:18:43.109553 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:18:43.109514    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:19:10.096961 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:10.096927    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:19:20.730284 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:20.730245    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:19:29.903186 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:29.903143    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:19:32.491524 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.491486    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"]
Sep 16 14:19:32.494801 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.494783    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.497233 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.497209    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 16 14:19:32.506269 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.506246    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"]
Sep 16 14:19:32.586622 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.586587    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc6gm\" (UniqueName: \"kubernetes.io/projected/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-kube-api-access-nc6gm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.586771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.586632    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.586771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.586672    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.586771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.586690    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.586771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.586737    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.586935 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.586818    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.687163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.687128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc6gm\" (UniqueName: \"kubernetes.io/projected/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-kube-api-access-nc6gm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.687163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.687169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.687417 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.687192    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.687417 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.687210    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.687417 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.687234    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.687417 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.687273    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.687709 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.687685    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.687772 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.687755    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.687821 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.687798    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.689719 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.689696    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.689950 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.689930    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.695382 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.695359    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc6gm\" (UniqueName: \"kubernetes.io/projected/c8ddb5f3-c3dc-4846-a2a0-0546f8301038-kube-api-access-nc6gm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbffm\" (UID: \"c8ddb5f3-c3dc-4846-a2a0-0546f8301038\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:32.805439 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:32.805366    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:33.081194 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:33.081118    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"]
Sep 16 14:19:33.084540 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:19:33.084508    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ddb5f3_c3dc_4846_a2a0_0546f8301038.slice/crio-9144ed78100c2b13f02ad7868868ecb476edb8d3612d425f9c1f76b276ca7313 WatchSource:0}: Error finding container 9144ed78100c2b13f02ad7868868ecb476edb8d3612d425f9c1f76b276ca7313: Status 404 returned error can't find the container with id 9144ed78100c2b13f02ad7868868ecb476edb8d3612d425f9c1f76b276ca7313
Sep 16 14:19:33.805249 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:33.805211    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm" event={"ID":"c8ddb5f3-c3dc-4846-a2a0-0546f8301038","Type":"ContainerStarted","Data":"2c4539a01adac78458ff69014df06513493222dfe489d3bd52a25f746e50f225"}
Sep 16 14:19:33.805657 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:33.805304    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm" event={"ID":"c8ddb5f3-c3dc-4846-a2a0-0546f8301038","Type":"ContainerStarted","Data":"9144ed78100c2b13f02ad7868868ecb476edb8d3612d425f9c1f76b276ca7313"}
Sep 16 14:19:33.828227 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:33.828176    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm" podStartSLOduration=1.636794895 podStartE2EDuration="1.828161309s" totalImagesPullingTime="191.366414ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:19:33.825191149 +0000 UTC m=+693.302423247" watchObservedRunningTime="2026-09-16 14:19:33.828161309 +0000 UTC m=+693.305393383"
Sep 16 14:19:37.789219 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.789184    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"]
Sep 16 14:19:37.793043 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.793023    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:37.795462 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.795442    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 16 14:19:37.802885 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.802863    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"]
Sep 16 14:19:37.929166 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.929126    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4dpl\" (UniqueName: \"kubernetes.io/projected/e03954e0-7648-49a3-959e-83632e7260d4-kube-api-access-x4dpl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:37.929166 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.929168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:37.929414 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.929192    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:37.929414 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.929256    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03954e0-7648-49a3-959e-83632e7260d4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:37.929414 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.929332    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:37.929414 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:37.929357    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.030205 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.030164    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4dpl\" (UniqueName: \"kubernetes.io/projected/e03954e0-7648-49a3-959e-83632e7260d4-kube-api-access-x4dpl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.030205 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.030204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.030474 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.030241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.030474 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.030270    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03954e0-7648-49a3-959e-83632e7260d4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.030474 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.030309    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.030474 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.030331    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.030750 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.030729    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.030811 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.030774    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.030867 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.030816    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.032909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.032885    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03954e0-7648-49a3-959e-83632e7260d4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.033139 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.033124    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03954e0-7648-49a3-959e-83632e7260d4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.040077 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.040017    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4dpl\" (UniqueName: \"kubernetes.io/projected/e03954e0-7648-49a3-959e-83632e7260d4-kube-api-access-x4dpl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-s29nw\" (UID: \"e03954e0-7648-49a3-959e-83632e7260d4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.103701 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.103667    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:38.375101 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.375067    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"]
Sep 16 14:19:38.377314 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:19:38.377281    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode03954e0_7648_49a3_959e_83632e7260d4.slice/crio-231ca75437ebd9160780518d30c4d565cad9350dd67dd7d0a6282f750c0d89a0 WatchSource:0}: Error finding container 231ca75437ebd9160780518d30c4d565cad9350dd67dd7d0a6282f750c0d89a0: Status 404 returned error can't find the container with id 231ca75437ebd9160780518d30c4d565cad9350dd67dd7d0a6282f750c0d89a0
Sep 16 14:19:38.828329 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.828295    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw" event={"ID":"e03954e0-7648-49a3-959e-83632e7260d4","Type":"ContainerStarted","Data":"c188377dca6604c0055403c986b6a84c2c71983d2f29649b5ba450bb01ebaa88"}
Sep 16 14:19:38.828722 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.828349    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw" event={"ID":"e03954e0-7648-49a3-959e-83632e7260d4","Type":"ContainerStarted","Data":"231ca75437ebd9160780518d30c4d565cad9350dd67dd7d0a6282f750c0d89a0"}
Sep 16 14:19:38.848175 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:38.848126    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw" podStartSLOduration=1.653714764 podStartE2EDuration="1.848109527s" totalImagesPullingTime="194.394763ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:19:38.847363832 +0000 UTC m=+698.324595918" watchObservedRunningTime="2026-09-16 14:19:38.848109527 +0000 UTC m=+698.325341601"
Sep 16 14:19:39.584212 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:39.584175    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:19:42.806419 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:42.806383    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:42.806852 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:42.806444    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:42.819073 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:42.819050    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:42.864396 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:42.864366    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbffm"
Sep 16 14:19:48.104423 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:48.104382    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:48.104841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:48.104462    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:48.117041 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:48.117011    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:19:48.889847 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:19:48.889818    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-s29nw"
Sep 16 14:20:58.949681 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:58.949650    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6cdfbdcc49-tqltg"]
Sep 16 14:20:58.952709 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:58.952689    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cdfbdcc49-tqltg"
Sep 16 14:20:58.955914 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:58.955892    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-slcv4\""
Sep 16 14:20:58.956013 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:58.955923    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 14:20:58.967749 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:58.967730    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cdfbdcc49-tqltg"]
Sep 16 14:20:59.013408 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:59.013371    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba836002-b083-4a42-868b-b9d4a61b0590-tls-cert\") pod \"authorino-6cdfbdcc49-tqltg\" (UID: \"ba836002-b083-4a42-868b-b9d4a61b0590\") " pod="kuadrant-system/authorino-6cdfbdcc49-tqltg"
Sep 16 14:20:59.013619 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:59.013491    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6p2b\" (UniqueName: \"kubernetes.io/projected/ba836002-b083-4a42-868b-b9d4a61b0590-kube-api-access-n6p2b\") pod \"authorino-6cdfbdcc49-tqltg\" (UID: \"ba836002-b083-4a42-868b-b9d4a61b0590\") " pod="kuadrant-system/authorino-6cdfbdcc49-tqltg"
Sep 16 14:20:59.114846 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:59.114809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6p2b\" (UniqueName: \"kubernetes.io/projected/ba836002-b083-4a42-868b-b9d4a61b0590-kube-api-access-n6p2b\") pod \"authorino-6cdfbdcc49-tqltg\" (UID: \"ba836002-b083-4a42-868b-b9d4a61b0590\") " pod="kuadrant-system/authorino-6cdfbdcc49-tqltg"
Sep 16 14:20:59.114846 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:59.114853    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba836002-b083-4a42-868b-b9d4a61b0590-tls-cert\") pod \"authorino-6cdfbdcc49-tqltg\" (UID: \"ba836002-b083-4a42-868b-b9d4a61b0590\") " pod="kuadrant-system/authorino-6cdfbdcc49-tqltg"
Sep 16 14:20:59.117511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:59.117487    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba836002-b083-4a42-868b-b9d4a61b0590-tls-cert\") pod \"authorino-6cdfbdcc49-tqltg\" (UID: \"ba836002-b083-4a42-868b-b9d4a61b0590\") " pod="kuadrant-system/authorino-6cdfbdcc49-tqltg"
Sep 16 14:20:59.125238 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:59.125218    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6p2b\" (UniqueName: \"kubernetes.io/projected/ba836002-b083-4a42-868b-b9d4a61b0590-kube-api-access-n6p2b\") pod \"authorino-6cdfbdcc49-tqltg\" (UID: \"ba836002-b083-4a42-868b-b9d4a61b0590\") " pod="kuadrant-system/authorino-6cdfbdcc49-tqltg"
Sep 16 14:20:59.261303 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:59.261213    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cdfbdcc49-tqltg"
Sep 16 14:20:59.534173 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:20:59.534150    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cdfbdcc49-tqltg"]
Sep 16 14:20:59.536999 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:20:59.536970    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba836002_b083_4a42_868b_b9d4a61b0590.slice/crio-a9cd9278dbc47ec7b629efded9d4bedf4aaaa13c44bb4c7c463d7251ec2405d3 WatchSource:0}: Error finding container a9cd9278dbc47ec7b629efded9d4bedf4aaaa13c44bb4c7c463d7251ec2405d3: Status 404 returned error can't find the container with id a9cd9278dbc47ec7b629efded9d4bedf4aaaa13c44bb4c7c463d7251ec2405d3
Sep 16 14:21:00.179730 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:21:00.179692    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cdfbdcc49-tqltg" event={"ID":"ba836002-b083-4a42-868b-b9d4a61b0590","Type":"ContainerStarted","Data":"a9cd9278dbc47ec7b629efded9d4bedf4aaaa13c44bb4c7c463d7251ec2405d3"}
Sep 16 14:21:01.185513 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:21:01.185477    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cdfbdcc49-tqltg" event={"ID":"ba836002-b083-4a42-868b-b9d4a61b0590","Type":"ContainerStarted","Data":"b297a4e0cfd69a101a8ddf4a0ed0209e8ee4df99aca063125f6ccba739459334"}
Sep 16 14:21:01.205502 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:21:01.205442    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6cdfbdcc49-tqltg" podStartSLOduration=2.367645194 podStartE2EDuration="3.205422554s" totalImagesPullingTime="837.77736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:21:01.204630887 +0000 UTC m=+780.681862961" watchObservedRunningTime="2026-09-16 14:21:01.205422554 +0000 UTC m=+780.682654629"
Sep 16 14:22:16.235187 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:16.235148    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:22:17.859046 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:17.859018    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"]
Sep 16 14:22:17.862728 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:17.862712    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"]
Sep 16 14:22:17.862845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:17.862825    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:17.865664 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:17.865647    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:17.866728 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:17.866701    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-c37e6e-dockercfg-hjczl\""
Sep 16 14:22:17.866864 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:17.866839    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-c37e6e\""
Sep 16 14:22:17.889126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:17.889105    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"]
Sep 16 14:22:17.897561 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:17.897541    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"]
Sep 16 14:22:18.000299 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.000264    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acccdabf-411a-4cc5-9c62-9add0560b775-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7\" (UID: \"acccdabf-411a-4cc5-9c62-9add0560b775\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:18.000486 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.000314    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6pps\" (UniqueName: \"kubernetes.io/projected/acccdabf-411a-4cc5-9c62-9add0560b775-kube-api-access-l6pps\") pod \"payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7\" (UID: \"acccdabf-411a-4cc5-9c62-9add0560b775\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:18.000486 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.000429    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/253f179e-81c3-4dda-8076-1d22a64fc595-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2\" (UID: \"253f179e-81c3-4dda-8076-1d22a64fc595\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:18.000486 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.000452    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47wbp\" (UniqueName: \"kubernetes.io/projected/253f179e-81c3-4dda-8076-1d22a64fc595-kube-api-access-47wbp\") pod \"payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2\" (UID: \"253f179e-81c3-4dda-8076-1d22a64fc595\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:18.101038 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.101007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/253f179e-81c3-4dda-8076-1d22a64fc595-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2\" (UID: \"253f179e-81c3-4dda-8076-1d22a64fc595\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:18.101038 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.101036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47wbp\" (UniqueName: \"kubernetes.io/projected/253f179e-81c3-4dda-8076-1d22a64fc595-kube-api-access-47wbp\") pod \"payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2\" (UID: \"253f179e-81c3-4dda-8076-1d22a64fc595\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:18.101219 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.101061    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acccdabf-411a-4cc5-9c62-9add0560b775-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7\" (UID: \"acccdabf-411a-4cc5-9c62-9add0560b775\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:18.101219 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.101121    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6pps\" (UniqueName: \"kubernetes.io/projected/acccdabf-411a-4cc5-9c62-9add0560b775-kube-api-access-l6pps\") pod \"payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7\" (UID: \"acccdabf-411a-4cc5-9c62-9add0560b775\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:18.101944 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.101924    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acccdabf-411a-4cc5-9c62-9add0560b775-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7\" (UID: \"acccdabf-411a-4cc5-9c62-9add0560b775\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:18.101991 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.101940    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/253f179e-81c3-4dda-8076-1d22a64fc595-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2\" (UID: \"253f179e-81c3-4dda-8076-1d22a64fc595\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:18.116243 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.116187    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6pps\" (UniqueName: \"kubernetes.io/projected/acccdabf-411a-4cc5-9c62-9add0560b775-kube-api-access-l6pps\") pod \"payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7\" (UID: \"acccdabf-411a-4cc5-9c62-9add0560b775\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:18.117958 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.117939    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47wbp\" (UniqueName: \"kubernetes.io/projected/253f179e-81c3-4dda-8076-1d22a64fc595-kube-api-access-47wbp\") pod \"payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2\" (UID: \"253f179e-81c3-4dda-8076-1d22a64fc595\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:18.173154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.173129    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:18.179292 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.179275    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:18.454999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.454971    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"]
Sep 16 14:22:18.457118 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:22:18.457090    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacccdabf_411a_4cc5_9c62_9add0560b775.slice/crio-426cfb0c4d406b8b2c73d84f78ac3684746a932c593e2bab3e2fcd10fb75582e WatchSource:0}: Error finding container 426cfb0c4d406b8b2c73d84f78ac3684746a932c593e2bab3e2fcd10fb75582e: Status 404 returned error can't find the container with id 426cfb0c4d406b8b2c73d84f78ac3684746a932c593e2bab3e2fcd10fb75582e
Sep 16 14:22:18.458128 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.458101    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"]
Sep 16 14:22:18.460621 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:22:18.460600    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253f179e_81c3_4dda_8076_1d22a64fc595.slice/crio-dca3dcb1d24de89e957554e07438464c30d0839a8be4656c9ea03519726ac4d4 WatchSource:0}: Error finding container dca3dcb1d24de89e957554e07438464c30d0839a8be4656c9ea03519726ac4d4: Status 404 returned error can't find the container with id dca3dcb1d24de89e957554e07438464c30d0839a8be4656c9ea03519726ac4d4
Sep 16 14:22:18.525017 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.524981    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2" event={"ID":"253f179e-81c3-4dda-8076-1d22a64fc595","Type":"ContainerStarted","Data":"dca3dcb1d24de89e957554e07438464c30d0839a8be4656c9ea03519726ac4d4"}
Sep 16 14:22:18.526153 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:18.526131    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7" event={"ID":"acccdabf-411a-4cc5-9c62-9add0560b775","Type":"ContainerStarted","Data":"426cfb0c4d406b8b2c73d84f78ac3684746a932c593e2bab3e2fcd10fb75582e"}
Sep 16 14:22:20.537919 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:20.537881    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2" event={"ID":"253f179e-81c3-4dda-8076-1d22a64fc595","Type":"ContainerStarted","Data":"1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5"}
Sep 16 14:22:20.538353 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:20.538095    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:20.539582 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:20.539541    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7" event={"ID":"acccdabf-411a-4cc5-9c62-9add0560b775","Type":"ContainerStarted","Data":"335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9"}
Sep 16 14:22:20.539713 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:20.539698    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:20.614755 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:20.614706    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2" podStartSLOduration=2.244393876 podStartE2EDuration="3.614691003s" totalImagesPullingTime="1.370297127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:20.6138096 +0000 UTC m=+860.091041697" watchObservedRunningTime="2026-09-16 14:22:20.614691003 +0000 UTC m=+860.091923078"
Sep 16 14:22:30.125223 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:30.125172    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7" podStartSLOduration=11.754216502 podStartE2EDuration="13.125156339s" totalImagesPullingTime="1.370939837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:20.659879874 +0000 UTC m=+860.137111978" watchObservedRunningTime="2026-09-16 14:22:30.125156339 +0000 UTC m=+869.602388413"
Sep 16 14:22:30.126791 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:30.126774    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:22:31.545755 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:31.545719    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:22:31.546136 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:31.545799    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:22:49.552480 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.552446    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"]
Sep 16 14:22:49.556137 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.556116    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.558854 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.558834    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"simulator-w3-d48266-kserve-self-signed-certs\""
Sep 16 14:22:49.559652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.559632    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"openshift-service-ca.crt\""
Sep 16 14:22:49.559722 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.559654    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"default-dockercfg-fdz6n\""
Sep 16 14:22:49.566556 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.566538    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"kube-root-ca.crt\""
Sep 16 14:22:49.591694 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.591671    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"]
Sep 16 14:22:49.687871 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.687845    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-dshm\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.688026 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.687893    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b67adb9-90d6-4269-b109-084963bae30e-tls-certs\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.688026 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.687971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-tmp-dir\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.688124 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.688037    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phqqq\" (UniqueName: \"kubernetes.io/projected/6b67adb9-90d6-4269-b109-084963bae30e-kube-api-access-phqqq\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.688124 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.688082    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-model-cache\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.688124 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.688103    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-home\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.788495 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.788464    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-home\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.788495 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.788497    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-dshm\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.788709 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.788520    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b67adb9-90d6-4269-b109-084963bae30e-tls-certs\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.788709 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.788553    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-tmp-dir\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.788709 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.788695    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phqqq\" (UniqueName: \"kubernetes.io/projected/6b67adb9-90d6-4269-b109-084963bae30e-kube-api-access-phqqq\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.788837 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.788749    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-model-cache\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.789015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.788980    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-home\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.789015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.789002    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-tmp-dir\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.789194 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.789029    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-model-cache\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.790770 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.790752    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-dshm\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.791180 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.791162    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b67adb9-90d6-4269-b109-084963bae30e-tls-certs\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.812532 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.812473    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phqqq\" (UniqueName: \"kubernetes.io/projected/6b67adb9-90d6-4269-b109-084963bae30e-kube-api-access-phqqq\") pod \"simulator-w3-d48266-kserve-768d8858c4-w8khn\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") " pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.893209 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.893174    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:49.899378 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.899354    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"]
Sep 16 14:22:49.902796 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.902773    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:22:49.905466 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.905439    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-0adaea-dockercfg-5sz7b\""
Sep 16 14:22:49.905654 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.905638    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-0adaea\""
Sep 16 14:22:49.916817 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.916794    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"]
Sep 16 14:22:49.947178 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.947108    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"]
Sep 16 14:22:49.950910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.950889    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:22:49.969010 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.968922    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"]
Sep 16 14:22:49.989912 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.989878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxpnv\" (UniqueName: \"kubernetes.io/projected/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-kube-api-access-dxpnv\") pod \"payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz\" (UID: \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:22:49.990059 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:49.990022    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz\" (UID: \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:22:50.091309 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.091231    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv\" (UID: \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:22:50.091309 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.091279    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxpnv\" (UniqueName: \"kubernetes.io/projected/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-kube-api-access-dxpnv\") pod \"payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz\" (UID: \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:22:50.091488 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.091370    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnnhs\" (UniqueName: \"kubernetes.io/projected/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-kube-api-access-mnnhs\") pod \"payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv\" (UID: \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:22:50.091488 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.091436    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz\" (UID: \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:22:50.092152 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.092132    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz\" (UID: \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:22:50.109436 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.109418    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxpnv\" (UniqueName: \"kubernetes.io/projected/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-kube-api-access-dxpnv\") pod \"payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz\" (UID: \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:22:50.191869 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.191835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnnhs\" (UniqueName: \"kubernetes.io/projected/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-kube-api-access-mnnhs\") pod \"payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv\" (UID: \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:22:50.192015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.191913    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv\" (UID: \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:22:50.192519 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.192503    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv\" (UID: \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:22:50.215593 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.213930    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:22:50.216641 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.216613    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"]
Sep 16 14:22:50.222719 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.222697    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnnhs\" (UniqueName: \"kubernetes.io/projected/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-kube-api-access-mnnhs\") pod \"payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv\" (UID: \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:22:50.265179 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.265135    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:22:50.517680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.517635    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"]
Sep 16 14:22:50.520160 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:22:50.520124    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0591d7d3_b995_4ae3_9664_22d5fc8aad3a.slice/crio-c24a08e6d22f6b98633ae4f8252e62864f47cd2d5dbec7d23c4af8d04f9ab83c WatchSource:0}: Error finding container c24a08e6d22f6b98633ae4f8252e62864f47cd2d5dbec7d23c4af8d04f9ab83c: Status 404 returned error can't find the container with id c24a08e6d22f6b98633ae4f8252e62864f47cd2d5dbec7d23c4af8d04f9ab83c
Sep 16 14:22:50.579907 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.579877    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"]
Sep 16 14:22:50.582133 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:22:50.582110    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff9c51c_e043_4081_9cbc_43bf3fc73c8e.slice/crio-f4202f35f16a02cad06950d52db4066b944cf16fa78faa6e0086884c5d8b3b8f WatchSource:0}: Error finding container f4202f35f16a02cad06950d52db4066b944cf16fa78faa6e0086884c5d8b3b8f: Status 404 returned error can't find the container with id f4202f35f16a02cad06950d52db4066b944cf16fa78faa6e0086884c5d8b3b8f
Sep 16 14:22:50.672324 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.672286    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz" event={"ID":"0591d7d3-b995-4ae3-9664-22d5fc8aad3a","Type":"ContainerStarted","Data":"8ce089e61b3a845c4ddad0f8ad09ba1ed36cf248751bda828ccdbaaa32e49ff1"}
Sep 16 14:22:50.672496 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.672354    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz" event={"ID":"0591d7d3-b995-4ae3-9664-22d5fc8aad3a","Type":"ContainerStarted","Data":"c24a08e6d22f6b98633ae4f8252e62864f47cd2d5dbec7d23c4af8d04f9ab83c"}
Sep 16 14:22:50.672496 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.672442    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:22:50.674174 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.674144    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn" event={"ID":"6b67adb9-90d6-4269-b109-084963bae30e","Type":"ContainerStarted","Data":"4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a"}
Sep 16 14:22:50.674305 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.674183    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn" event={"ID":"6b67adb9-90d6-4269-b109-084963bae30e","Type":"ContainerStarted","Data":"d9004ac381316c8d3aeb0d91314ce6c4c3da30627a68059cd552c022b15fd2e5"}
Sep 16 14:22:50.675841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.675815    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv" event={"ID":"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e","Type":"ContainerStarted","Data":"bc626399a7dbaa8a8126dd18b23996d83471d68d313ae7c29375dfdef27f1c40"}
Sep 16 14:22:50.675935 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.675868    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv" event={"ID":"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e","Type":"ContainerStarted","Data":"f4202f35f16a02cad06950d52db4066b944cf16fa78faa6e0086884c5d8b3b8f"}
Sep 16 14:22:50.675935 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.675897    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:22:50.728739 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.728674    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz" podStartSLOduration=1.728650251 podStartE2EDuration="1.728650251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:50.72726018 +0000 UTC m=+890.204492274" watchObservedRunningTime="2026-09-16 14:22:50.728650251 +0000 UTC m=+890.205882309"
Sep 16 14:22:50.784970 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:50.784874    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv" podStartSLOduration=1.784855817 podStartE2EDuration="1.784855817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:50.773693404 +0000 UTC m=+890.250925478" watchObservedRunningTime="2026-09-16 14:22:50.784855817 +0000 UTC m=+890.262087892"
Sep 16 14:22:55.434161 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:55.434051    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn" podStartSLOduration=6.434035271 podStartE2EDuration="6.434035271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:50.83769092 +0000 UTC m=+890.314923023" watchObservedRunningTime="2026-09-16 14:22:55.434035271 +0000 UTC m=+894.911267345"
Sep 16 14:22:55.434872 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:55.434851    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:22:59.893430 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:59.893390    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:59.893846 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:59.893446    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:22:59.905831 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:22:59.905808    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:23:00.734762 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:00.734733    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:23:01.683286 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:01.683246    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:23:01.683782 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:01.683344    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:23:06.721186 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.721154    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"]
Sep 16 14:23:06.724393 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.724374    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.729342 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.729322    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"default-dockercfg-rnmfp\""
Sep 16 14:23:06.730039 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.730021    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"openshift-service-ca.crt\""
Sep 16 14:23:06.730117 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.730055    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"kube-root-ca.crt\""
Sep 16 14:23:06.730336 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.730319    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"simulator-w0-11636c-kserve-self-signed-certs\""
Sep 16 14:23:06.734829 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.734810    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-model-cache\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.734921 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.734834    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-tmp-dir\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.734921 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.734858    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-home\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.734921 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.734887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-dshm\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.734921 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.734912    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/004dda30-a002-46e8-8322-c5f543826458-tls-certs\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.735068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.734978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-996jj\" (UniqueName: \"kubernetes.io/projected/004dda30-a002-46e8-8322-c5f543826458-kube-api-access-996jj\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.742727 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.742693    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"]
Sep 16 14:23:06.836362 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.836332    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-996jj\" (UniqueName: \"kubernetes.io/projected/004dda30-a002-46e8-8322-c5f543826458-kube-api-access-996jj\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.836499 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.836384    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-model-cache\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.836499 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.836406    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-tmp-dir\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.836499 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.836440    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-home\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.836499 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.836479    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-dshm\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.836695 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.836500    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/004dda30-a002-46e8-8322-c5f543826458-tls-certs\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.836856 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.836835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-model-cache\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.836895 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.836850    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-tmp-dir\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.836948 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.836926    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-home\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.838835 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.838811    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-dshm\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.839152 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.839133    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/004dda30-a002-46e8-8322-c5f543826458-tls-certs\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:06.845796 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:06.845776    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-996jj\" (UniqueName: \"kubernetes.io/projected/004dda30-a002-46e8-8322-c5f543826458-kube-api-access-996jj\") pod \"simulator-w0-11636c-kserve-7ddd965c59-q5mrs\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") " pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:07.033359 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:07.033271    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:07.318353 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:07.318294    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"]
Sep 16 14:23:07.320608 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:23:07.320578    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod004dda30_a002_46e8_8322_c5f543826458.slice/crio-a84ddcb58b0ea8db10dd4f0a9306c60dd81a0b84c853d384750b387990a204f6 WatchSource:0}: Error finding container a84ddcb58b0ea8db10dd4f0a9306c60dd81a0b84c853d384750b387990a204f6: Status 404 returned error can't find the container with id a84ddcb58b0ea8db10dd4f0a9306c60dd81a0b84c853d384750b387990a204f6
Sep 16 14:23:07.758895 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:07.758858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs" event={"ID":"004dda30-a002-46e8-8322-c5f543826458","Type":"ContainerStarted","Data":"4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295"}
Sep 16 14:23:07.759311 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:07.758910    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs" event={"ID":"004dda30-a002-46e8-8322-c5f543826458","Type":"ContainerStarted","Data":"a84ddcb58b0ea8db10dd4f0a9306c60dd81a0b84c853d384750b387990a204f6"}
Sep 16 14:23:07.786174 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:07.786127    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs" podStartSLOduration=1.7861123399999999 podStartE2EDuration="1.78611234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:07.782333925 +0000 UTC m=+907.259566024" watchObservedRunningTime="2026-09-16 14:23:07.78611234 +0000 UTC m=+907.263344412"
Sep 16 14:23:10.231967 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:10.231934    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:23:17.034137 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:17.034102    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:17.034538 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:17.034159    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:17.046803 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:17.046773    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:17.815706 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:17.815672    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:23:28.521858 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.521819    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"]
Sep 16 14:23:28.525640 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.525621    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.532386 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.532357    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c37e6e\"/\"kube-root-ca.crt\""
Sep 16 14:23:28.532512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.532384    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c37e6e\"/\"default-dockercfg-56czm\""
Sep 16 14:23:28.532512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.532368    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c37e6e\"/\"test-model-c37e6e-kserve-self-signed-certs\""
Sep 16 14:23:28.532512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.532368    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c37e6e\"/\"openshift-service-ca.crt\""
Sep 16 14:23:28.543728 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.543704    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"]
Sep 16 14:23:28.631903 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.631866    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-home\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.631903 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.631901    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcb5d\" (UniqueName: \"kubernetes.io/projected/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-kube-api-access-dcb5d\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.632105 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.631941    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tls-certs\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.632105 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.631986    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-model-cache\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.632105 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.632026    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tmp-dir\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.632105 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.632050    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-dshm\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.733128 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.733083    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-home\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.733128 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.733127    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcb5d\" (UniqueName: \"kubernetes.io/projected/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-kube-api-access-dcb5d\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.733329 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.733159    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tls-certs\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.733329 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.733188    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-model-cache\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.733329 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.733227    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tmp-dir\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.733329 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.733261    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-dshm\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.733545 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.733522    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-home\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.733652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.733635    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tmp-dir\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.733720 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.733641    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-model-cache\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.735710 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.735690    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-dshm\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.735871 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.735856    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tls-certs\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.743148 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.743128    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcb5d\" (UniqueName: \"kubernetes.io/projected/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-kube-api-access-dcb5d\") pod \"test-model-c37e6e-kserve-b8c585fb7-tthk5\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") " pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:28.836479 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:28.836400    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:29.124209 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:29.124179    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"]
Sep 16 14:23:29.125999 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:23:29.125975    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc8abfd8_9bea_4b97_a6b5_9d93cb05705c.slice/crio-0513ca7cd83921a296c76d4d89b8b2ea7474774852e34e753d76e8bba36360a9 WatchSource:0}: Error finding container 0513ca7cd83921a296c76d4d89b8b2ea7474774852e34e753d76e8bba36360a9: Status 404 returned error can't find the container with id 0513ca7cd83921a296c76d4d89b8b2ea7474774852e34e753d76e8bba36360a9
Sep 16 14:23:29.862035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:29.861996    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5" event={"ID":"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c","Type":"ContainerStarted","Data":"0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3"}
Sep 16 14:23:29.862035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:29.862039    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5" event={"ID":"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c","Type":"ContainerStarted","Data":"0513ca7cd83921a296c76d4d89b8b2ea7474774852e34e753d76e8bba36360a9"}
Sep 16 14:23:29.888226 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:29.888176    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5" podStartSLOduration=1.88815938 podStartE2EDuration="1.88815938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:29.886611188 +0000 UTC m=+929.363843272" watchObservedRunningTime="2026-09-16 14:23:29.88815938 +0000 UTC m=+929.365391455"
Sep 16 14:23:34.929488 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:34.929448    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:23:38.836910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:38.836873    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:38.836910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:38.836920    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:38.849464 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:38.849436    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:38.915188 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:38.915154    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:23:53.028352 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.028318    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"]
Sep 16 14:23:53.031703 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.031686    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.035980 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.035955    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"premium-w3-d48266-kserve-self-signed-certs\""
Sep 16 14:23:53.052716 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.052694    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-home\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.052805 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.052738    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-model-cache\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.052861 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.052841    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-tmp-dir\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.052902 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.052875    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94b3006c-d63e-4618-b9ec-b6c54637b16f-tls-certs\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.052945 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.052908    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-dshm\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.052945 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.052935    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngfgm\" (UniqueName: \"kubernetes.io/projected/94b3006c-d63e-4618-b9ec-b6c54637b16f-kube-api-access-ngfgm\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.062023 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.062002    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"]
Sep 16 14:23:53.154140 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.154106    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-home\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.154302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.154167    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-model-cache\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.154302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.154219    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-tmp-dir\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.154302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.154241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94b3006c-d63e-4618-b9ec-b6c54637b16f-tls-certs\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.154302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.154266    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-dshm\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.154302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.154287    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngfgm\" (UniqueName: \"kubernetes.io/projected/94b3006c-d63e-4618-b9ec-b6c54637b16f-kube-api-access-ngfgm\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.154547 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.154504    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-home\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.154638 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.154559    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-model-cache\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.154709 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.154631    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-tmp-dir\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.156750 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.156723    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-dshm\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.156959 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.156941    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94b3006c-d63e-4618-b9ec-b6c54637b16f-tls-certs\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.165700 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.165676    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngfgm\" (UniqueName: \"kubernetes.io/projected/94b3006c-d63e-4618-b9ec-b6c54637b16f-kube-api-access-ngfgm\") pod \"premium-w3-d48266-kserve-645ff5669f-pwgc4\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") " pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.341769 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.341676    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:23:53.628201 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.628178    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"]
Sep 16 14:23:53.630401 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:23:53.630375    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94b3006c_d63e_4618_b9ec_b6c54637b16f.slice/crio-ea63ccecd39c95d73fdd20a7d412363ac1a4df14ccdee4c0c81550150538bd70 WatchSource:0}: Error finding container ea63ccecd39c95d73fdd20a7d412363ac1a4df14ccdee4c0c81550150538bd70: Status 404 returned error can't find the container with id ea63ccecd39c95d73fdd20a7d412363ac1a4df14ccdee4c0c81550150538bd70
Sep 16 14:23:53.988295 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.988265    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4" event={"ID":"94b3006c-d63e-4618-b9ec-b6c54637b16f","Type":"ContainerStarted","Data":"c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840"}
Sep 16 14:23:53.988460 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:53.988306    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4" event={"ID":"94b3006c-d63e-4618-b9ec-b6c54637b16f","Type":"ContainerStarted","Data":"ea63ccecd39c95d73fdd20a7d412363ac1a4df14ccdee4c0c81550150538bd70"}
Sep 16 14:23:54.014151 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:54.014101    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4" podStartSLOduration=2.014086586 podStartE2EDuration="2.014086586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:54.012783513 +0000 UTC m=+953.490015649" watchObservedRunningTime="2026-09-16 14:23:54.014086586 +0000 UTC m=+953.491318660"
Sep 16 14:23:58.729877 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:23:58.729842    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:24:03.342748 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:03.342703    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:24:03.343344 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:03.342804    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:24:03.355633 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:03.355605    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:24:04.044484 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:04.044448    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:24:22.863147 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:22.863048    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:24:44.169589 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.169534    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:24:44.775879 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.775849    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"]
Sep 16 14:24:44.779244 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.779229    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.782350 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:24:44.782322    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-213.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-0adaea\": no relationship found between node 'ip-10-0-137-213.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-0adaea\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 16 14:24:44.783065 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:24:44.783027    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-model-0adaea-kserve-7b9b784669-rcznm\" is forbidden: User \"system:node:ip-10-0-137-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-0adaea\": no relationship found between node 'ip-10-0-137-213.ec2.internal' and this object" podUID="13d5673c-1465-4426-b6c7-230205396faa" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.783168 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:24:44.783083    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"test-model-0adaea-kserve-self-signed-certs\" is forbidden: User \"system:node:ip-10-0-137-213.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-0adaea\": no relationship found between node 'ip-10-0-137-213.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-0adaea\"/\"test-model-0adaea-kserve-self-signed-certs\"" type="*v1.Secret"
Sep 16 14:24:44.783932 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:24:44.783912    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-213.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-0adaea\": no relationship found between node 'ip-10-0-137-213.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-0adaea\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 16 14:24:44.807041 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:24:44.807010    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-xhd2q\" is forbidden: User \"system:node:ip-10-0-137-213.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-0adaea\": no relationship found between node 'ip-10-0-137-213.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-0adaea\"/\"default-dockercfg-xhd2q\"" type="*v1.Secret"
Sep 16 14:24:44.811187 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.811159    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-tmp-dir\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.811297 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.811211    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-model-cache\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.811297 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.811236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13d5673c-1465-4426-b6c7-230205396faa-tls-certs\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.811297 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.811257    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-dshm\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.811297 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.811275    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl8q9\" (UniqueName: \"kubernetes.io/projected/13d5673c-1465-4426-b6c7-230205396faa-kube-api-access-dl8q9\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.811512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.811353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-home\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.812045 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.812021    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"]
Sep 16 14:24:44.912760 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.912722    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-tmp-dir\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.912911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.912788    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-model-cache\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.912911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.912813    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13d5673c-1465-4426-b6c7-230205396faa-tls-certs\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.912911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.912842    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-dshm\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.912911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.912873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8q9\" (UniqueName: \"kubernetes.io/projected/13d5673c-1465-4426-b6c7-230205396faa-kube-api-access-dl8q9\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.913070 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.912926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-home\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.913182 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.913165    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-tmp-dir\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.913232 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.913210    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-model-cache\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.913274 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.913254    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-home\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:44.915195 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:44.915177    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-dshm\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:45.814016 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:45.813982    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0adaea\"/\"kube-root-ca.crt\""
Sep 16 14:24:45.877416 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:45.877385    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0adaea\"/\"openshift-service-ca.crt\""
Sep 16 14:24:45.890088 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:45.890063    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl8q9\" (UniqueName: \"kubernetes.io/projected/13d5673c-1465-4426-b6c7-230205396faa-kube-api-access-dl8q9\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:45.913052 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:24:45.913023    2566 secret.go:189] Couldn't get secret ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-self-signed-certs: failed to sync secret cache: timed out waiting for the condition
Sep 16 14:24:45.913212 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:24:45.913119    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13d5673c-1465-4426-b6c7-230205396faa-tls-certs podName:13d5673c-1465-4426-b6c7-230205396faa nodeName:}" failed. No retries permitted until 2026-09-16 14:24:46.413098454 +0000 UTC m=+1005.890330517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/13d5673c-1465-4426-b6c7-230205396faa-tls-certs") pod "test-model-0adaea-kserve-7b9b784669-rcznm" (UID: "13d5673c-1465-4426-b6c7-230205396faa") : failed to sync secret cache: timed out waiting for the condition
Sep 16 14:24:45.955308 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:45.955278    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0adaea\"/\"test-model-0adaea-kserve-self-signed-certs\""
Sep 16 14:24:46.004539 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:46.004512    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0adaea\"/\"default-dockercfg-xhd2q\""
Sep 16 14:24:46.427191 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:46.427157    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13d5673c-1465-4426-b6c7-230205396faa-tls-certs\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:46.429756 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:46.429727    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13d5673c-1465-4426-b6c7-230205396faa-tls-certs\") pod \"test-model-0adaea-kserve-7b9b784669-rcznm\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") " pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:46.588365 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:46.588325    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:46.896948 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:46.896917    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"]
Sep 16 14:24:46.898656 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:24:46.898631    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13d5673c_1465_4426_b6c7_230205396faa.slice/crio-1199b0b3105173822e0c6dcb5f4352b21bdf7682abb0cfe13d0c1f95f6528121 WatchSource:0}: Error finding container 1199b0b3105173822e0c6dcb5f4352b21bdf7682abb0cfe13d0c1f95f6528121: Status 404 returned error can't find the container with id 1199b0b3105173822e0c6dcb5f4352b21bdf7682abb0cfe13d0c1f95f6528121
Sep 16 14:24:47.228578 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:47.228539    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm" event={"ID":"13d5673c-1465-4426-b6c7-230205396faa","Type":"ContainerStarted","Data":"7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22"}
Sep 16 14:24:47.228755 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:47.228606    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm" event={"ID":"13d5673c-1465-4426-b6c7-230205396faa","Type":"ContainerStarted","Data":"1199b0b3105173822e0c6dcb5f4352b21bdf7682abb0cfe13d0c1f95f6528121"}
Sep 16 14:24:47.283059 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:47.283002    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm" podStartSLOduration=3.282981528 podStartE2EDuration="3.282981528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:24:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:24:47.281878811 +0000 UTC m=+1006.759110886" watchObservedRunningTime="2026-09-16 14:24:47.282981528 +0000 UTC m=+1006.760213603"
Sep 16 14:24:56.588652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:56.588612    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:56.589052 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:56.588681    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:56.603746 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:56.603721    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:24:57.286822 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:24:57.286790    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:25:09.437206 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:25:09.437172    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:25:41.091336 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:25:41.091294    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:26:05.493183 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.493105    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"]
Sep 16 14:26:05.497075 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.497054    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.499444 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.499415    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"distinct-w0-11636c-kserve-self-signed-certs\""
Sep 16 14:26:05.507161 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.507138    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"]
Sep 16 14:26:05.559002 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.558968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d367e6-f559-4c22-b974-f75e645b5fbc-tls-certs\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.559002 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.559011    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-model-cache\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.559268 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.559042    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-dshm\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.559268 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.559070    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-home\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.559268 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.559087    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj4qx\" (UniqueName: \"kubernetes.io/projected/a7d367e6-f559-4c22-b974-f75e645b5fbc-kube-api-access-kj4qx\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.559268 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.559150    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-tmp-dir\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.660090 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.660050    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-home\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.660090 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.660093    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj4qx\" (UniqueName: \"kubernetes.io/projected/a7d367e6-f559-4c22-b974-f75e645b5fbc-kube-api-access-kj4qx\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.660342 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.660160    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-tmp-dir\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.660342 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.660208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d367e6-f559-4c22-b974-f75e645b5fbc-tls-certs\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.660342 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.660245    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-model-cache\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.660342 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.660300    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-dshm\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.660545 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.660495    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-home\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.660545 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.660508    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-tmp-dir\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.660771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.660748    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-model-cache\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.662645 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.662624    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-dshm\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.663049 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.663025    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d367e6-f559-4c22-b974-f75e645b5fbc-tls-certs\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.668722 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.668693    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj4qx\" (UniqueName: \"kubernetes.io/projected/a7d367e6-f559-4c22-b974-f75e645b5fbc-kube-api-access-kj4qx\") pod \"distinct-w0-11636c-kserve-5db449746f-wmvt2\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:05.809964 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:05.809875    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:06.082805 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:06.082780    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"]
Sep 16 14:26:06.084947 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:26:06.084914    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7d367e6_f559_4c22_b974_f75e645b5fbc.slice/crio-a89cb5f5aea250192ff4e96905a3e6f9d4234388d1c205f0db2115b21f476e4b WatchSource:0}: Error finding container a89cb5f5aea250192ff4e96905a3e6f9d4234388d1c205f0db2115b21f476e4b: Status 404 returned error can't find the container with id a89cb5f5aea250192ff4e96905a3e6f9d4234388d1c205f0db2115b21f476e4b
Sep 16 14:26:06.606189 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:06.606155    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2" event={"ID":"a7d367e6-f559-4c22-b974-f75e645b5fbc","Type":"ContainerStarted","Data":"d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc"}
Sep 16 14:26:06.606189 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:06.606196    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2" event={"ID":"a7d367e6-f559-4c22-b974-f75e645b5fbc","Type":"ContainerStarted","Data":"a89cb5f5aea250192ff4e96905a3e6f9d4234388d1c205f0db2115b21f476e4b"}
Sep 16 14:26:06.627856 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:06.627811    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2" podStartSLOduration=1.627795828 podStartE2EDuration="1.627795828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:26:06.62659381 +0000 UTC m=+1086.103825879" watchObservedRunningTime="2026-09-16 14:26:06.627795828 +0000 UTC m=+1086.105027902"
Sep 16 14:26:12.000922 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:12.000884    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:26:12.081374 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:12.081336    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"]
Sep 16 14:26:15.810520 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:15.810483    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:15.810520 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:15.810526    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:15.823110 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:15.823081    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:16.666683 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:16.666649    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:26:27.101819 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.101781    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5" podUID="dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" containerName="main" containerID="cri-o://0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3" gracePeriod=45
Sep 16 14:26:27.344232 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.344208    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:26:27.458272 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458243    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tls-certs\") pod \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") "
Sep 16 14:26:27.458431 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458278    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-model-cache\") pod \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") "
Sep 16 14:26:27.458431 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458293    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-dshm\") pod \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") "
Sep 16 14:26:27.458541 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458487    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-home\") pod \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") "
Sep 16 14:26:27.458541 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458509    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-model-cache" (OuterVolumeSpecName: "model-cache") pod "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" (UID: "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:27.458677 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458554    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcb5d\" (UniqueName: \"kubernetes.io/projected/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-kube-api-access-dcb5d\") pod \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") "
Sep 16 14:26:27.458677 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458671    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tmp-dir\") pod \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\" (UID: \"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c\") "
Sep 16 14:26:27.458799 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458774    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-home" (OuterVolumeSpecName: "home") pod "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" (UID: "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:27.458967 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458946    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:27.458967 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458971    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:27.459106 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.458943    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" (UID: "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:27.460689 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.460661    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-dshm" (OuterVolumeSpecName: "dshm") pod "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" (UID: "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:27.460771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.460663    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" (UID: "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:26:27.460841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.460821    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-kube-api-access-dcb5d" (OuterVolumeSpecName: "kube-api-access-dcb5d") pod "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" (UID: "dc8abfd8-9bea-4b97-a6b5-9d93cb05705c"). InnerVolumeSpecName "kube-api-access-dcb5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:27.560335 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.560304    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcb5d\" (UniqueName: \"kubernetes.io/projected/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-kube-api-access-dcb5d\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:27.560335 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.560330    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:27.560516 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.560343    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:27.560516 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.560351    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:27.708668 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.708549    2566 generic.go:356] "Generic (PLEG): container finished" podID="dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" containerID="0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3" exitCode=0
Sep 16 14:26:27.708668 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.708618    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"
Sep 16 14:26:27.708668 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.708648    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5" event={"ID":"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c","Type":"ContainerDied","Data":"0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3"}
Sep 16 14:26:27.708938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.708693    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5" event={"ID":"dc8abfd8-9bea-4b97-a6b5-9d93cb05705c","Type":"ContainerDied","Data":"0513ca7cd83921a296c76d4d89b8b2ea7474774852e34e753d76e8bba36360a9"}
Sep 16 14:26:27.708938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.708714    2566 scope.go:117] "RemoveContainer" containerID="0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3"
Sep 16 14:26:27.717936 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.717919    2566 scope.go:117] "RemoveContainer" containerID="0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3"
Sep 16 14:26:27.718204 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:26:27.718186    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3\": container with ID starting with 0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3 not found: ID does not exist" containerID="0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3"
Sep 16 14:26:27.718251 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.718212    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3"} err="failed to get container status \"0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3\": rpc error: code = NotFound desc = could not find container \"0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3\": container with ID starting with 0ac0fac526454284b11744e5a48a781a3db4afc2061db4156872aeadf47ce1f3 not found: ID does not exist"
Sep 16 14:26:27.732406 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.732379    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"]
Sep 16 14:26:27.735208 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:27.735188    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c37e6e/test-model-c37e6e-kserve-b8c585fb7-tthk5"]
Sep 16 14:26:28.984968 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:28.984937    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" path="/var/lib/kubelet/pods/dc8abfd8-9bea-4b97-a6b5-9d93cb05705c/volumes"
Sep 16 14:26:43.074232 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.074201    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"]
Sep 16 14:26:43.074688 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.074549    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" containerName="main"
Sep 16 14:26:43.074688 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.074560    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" containerName="main"
Sep 16 14:26:43.074688 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.074645    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc8abfd8-9bea-4b97-a6b5-9d93cb05705c" containerName="main"
Sep 16 14:26:43.077905 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.077881    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"
Sep 16 14:26:43.080854 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.080829    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:26:43.081132 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.081115    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:26:43.089640 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.089622    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"]
Sep 16 14:26:43.207592 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.207533    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea81d5e-14d4-4028-b90b-c2bc1f965dcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr\" (UID: \"8ea81d5e-14d4-4028-b90b-c2bc1f965dcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"
Sep 16 14:26:43.308324 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.308283    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea81d5e-14d4-4028-b90b-c2bc1f965dcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr\" (UID: \"8ea81d5e-14d4-4028-b90b-c2bc1f965dcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"
Sep 16 14:26:43.308943 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.308923    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea81d5e-14d4-4028-b90b-c2bc1f965dcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr\" (UID: \"8ea81d5e-14d4-4028-b90b-c2bc1f965dcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"
Sep 16 14:26:43.389186 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.389082    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"
Sep 16 14:26:43.664668 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.664642    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"]
Sep 16 14:26:43.670080 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:26:43.670041    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea81d5e_14d4_4028_b90b_c2bc1f965dcb.slice/crio-58d7e8b60c151fb64abe523d95e3d52aaf751f2c025b955d6c856b1d966ff4af WatchSource:0}: Error finding container 58d7e8b60c151fb64abe523d95e3d52aaf751f2c025b955d6c856b1d966ff4af: Status 404 returned error can't find the container with id 58d7e8b60c151fb64abe523d95e3d52aaf751f2c025b955d6c856b1d966ff4af
Sep 16 14:26:43.672181 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.672166    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:26:43.794110 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:43.794076    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr" event={"ID":"8ea81d5e-14d4-4028-b90b-c2bc1f965dcb","Type":"ContainerStarted","Data":"58d7e8b60c151fb64abe523d95e3d52aaf751f2c025b955d6c856b1d966ff4af"}
Sep 16 14:26:46.812161 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:46.812125    2566 generic.go:356] "Generic (PLEG): container finished" podID="8ea81d5e-14d4-4028-b90b-c2bc1f965dcb" containerID="adb86038750e963e7f459068e6019eda024957ce1a5d6a50d20d89ade3c70733" exitCode=0
Sep 16 14:26:46.812586 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:46.812219    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr" event={"ID":"8ea81d5e-14d4-4028-b90b-c2bc1f965dcb","Type":"ContainerDied","Data":"adb86038750e963e7f459068e6019eda024957ce1a5d6a50d20d89ade3c70733"}
Sep 16 14:26:47.947956 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:47.947933    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"
Sep 16 14:26:48.050963 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:48.050159    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea81d5e-14d4-4028-b90b-c2bc1f965dcb-maas-api-service-ca\") pod \"8ea81d5e-14d4-4028-b90b-c2bc1f965dcb\" (UID: \"8ea81d5e-14d4-4028-b90b-c2bc1f965dcb\") "
Sep 16 14:26:48.053652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:48.051292    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea81d5e-14d4-4028-b90b-c2bc1f965dcb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ea81d5e-14d4-4028-b90b-c2bc1f965dcb" (UID: "8ea81d5e-14d4-4028-b90b-c2bc1f965dcb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:26:48.151447 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:48.151370    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea81d5e-14d4-4028-b90b-c2bc1f965dcb-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:48.823315 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:48.823282    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr" event={"ID":"8ea81d5e-14d4-4028-b90b-c2bc1f965dcb","Type":"ContainerDied","Data":"58d7e8b60c151fb64abe523d95e3d52aaf751f2c025b955d6c856b1d966ff4af"}
Sep 16 14:26:48.823315 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:48.823312    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"
Sep 16 14:26:48.823517 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:48.823357    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58d7e8b60c151fb64abe523d95e3d52aaf751f2c025b955d6c856b1d966ff4af"
Sep 16 14:26:49.269957 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:49.269926    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"]
Sep 16 14:26:49.716910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:49.716882    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:26:50.266397 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.266362    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"]
Sep 16 14:26:50.266755 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.266742    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ea81d5e-14d4-4028-b90b-c2bc1f965dcb" containerName="revoke-keys"
Sep 16 14:26:50.266812 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.266757    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ea81d5e-14d4-4028-b90b-c2bc1f965dcb" containerName="revoke-keys"
Sep 16 14:26:50.266850 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.266838    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ea81d5e-14d4-4028-b90b-c2bc1f965dcb" containerName="revoke-keys"
Sep 16 14:26:50.270256 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.270229    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"
Sep 16 14:26:50.272602 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.272555    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:26:50.272843 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.272827    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:26:50.296150 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.296126    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"]
Sep 16 14:26:50.371099 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.371061    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91179b4c-8576-4a08-b719-831530cb0f8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr\" (UID: \"91179b4c-8576-4a08-b719-831530cb0f8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"
Sep 16 14:26:50.471542 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.471507    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91179b4c-8576-4a08-b719-831530cb0f8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr\" (UID: \"91179b4c-8576-4a08-b719-831530cb0f8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"
Sep 16 14:26:50.472239 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.472217    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91179b4c-8576-4a08-b719-831530cb0f8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr\" (UID: \"91179b4c-8576-4a08-b719-831530cb0f8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"
Sep 16 14:26:50.580057 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.579983    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"
Sep 16 14:26:50.860356 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:50.860334    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"]
Sep 16 14:26:50.862738 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:26:50.862698    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91179b4c_8576_4a08_b719_831530cb0f8b.slice/crio-5dd58503e4f99a38f2bfc475ad7edcc5d5e392f1ad02158ef225b6570c37097e WatchSource:0}: Error finding container 5dd58503e4f99a38f2bfc475ad7edcc5d5e392f1ad02158ef225b6570c37097e: Status 404 returned error can't find the container with id 5dd58503e4f99a38f2bfc475ad7edcc5d5e392f1ad02158ef225b6570c37097e
Sep 16 14:26:51.838068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:51.838031    2566 generic.go:356] "Generic (PLEG): container finished" podID="91179b4c-8576-4a08-b719-831530cb0f8b" containerID="bc80e0e670d2d659ff75dcdd9c611d61083483c949baf1579f056a7195a2e11a" exitCode=0
Sep 16 14:26:51.838434 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:51.838119    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr" event={"ID":"91179b4c-8576-4a08-b719-831530cb0f8b","Type":"ContainerDied","Data":"bc80e0e670d2d659ff75dcdd9c611d61083483c949baf1579f056a7195a2e11a"}
Sep 16 14:26:51.838434 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:51.838170    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr" event={"ID":"91179b4c-8576-4a08-b719-831530cb0f8b","Type":"ContainerStarted","Data":"5dd58503e4f99a38f2bfc475ad7edcc5d5e392f1ad02158ef225b6570c37097e"}
Sep 16 14:26:52.978203 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:52.978172    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"
Sep 16 14:26:52.992304 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:52.992278    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91179b4c-8576-4a08-b719-831530cb0f8b-maas-api-service-ca\") pod \"91179b4c-8576-4a08-b719-831530cb0f8b\" (UID: \"91179b4c-8576-4a08-b719-831530cb0f8b\") "
Sep 16 14:26:52.992656 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:52.992637    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91179b4c-8576-4a08-b719-831530cb0f8b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91179b4c-8576-4a08-b719-831530cb0f8b" (UID: "91179b4c-8576-4a08-b719-831530cb0f8b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:26:53.093934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:53.093857    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91179b4c-8576-4a08-b719-831530cb0f8b-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:53.849907 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:53.849883    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"
Sep 16 14:26:53.850071 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:53.849883    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr" event={"ID":"91179b4c-8576-4a08-b719-831530cb0f8b","Type":"ContainerDied","Data":"5dd58503e4f99a38f2bfc475ad7edcc5d5e392f1ad02158ef225b6570c37097e"}
Sep 16 14:26:53.850071 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:53.849991    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5dd58503e4f99a38f2bfc475ad7edcc5d5e392f1ad02158ef225b6570c37097e"
Sep 16 14:26:55.226653 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.226623    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"]
Sep 16 14:26:55.227015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.226807    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2" podUID="253f179e-81c3-4dda-8076-1d22a64fc595" containerName="payload-processing" containerID="cri-o://1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5" gracePeriod=30
Sep 16 14:26:55.244128 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.244105    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"]
Sep 16 14:26:55.244322 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.244287    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7" podUID="acccdabf-411a-4cc5-9c62-9add0560b775" containerName="payload-pre-processing" containerID="cri-o://335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9" gracePeriod=30
Sep 16 14:26:55.515115 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.515089    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:26:55.518085 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.518067    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:26:55.615451 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.615419    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6pps\" (UniqueName: \"kubernetes.io/projected/acccdabf-411a-4cc5-9c62-9add0560b775-kube-api-access-l6pps\") pod \"acccdabf-411a-4cc5-9c62-9add0560b775\" (UID: \"acccdabf-411a-4cc5-9c62-9add0560b775\") "
Sep 16 14:26:55.615451 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.615457    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acccdabf-411a-4cc5-9c62-9add0560b775-plugins-config-volume\") pod \"acccdabf-411a-4cc5-9c62-9add0560b775\" (UID: \"acccdabf-411a-4cc5-9c62-9add0560b775\") "
Sep 16 14:26:55.615688 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.615524    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/253f179e-81c3-4dda-8076-1d22a64fc595-plugins-config-volume\") pod \"253f179e-81c3-4dda-8076-1d22a64fc595\" (UID: \"253f179e-81c3-4dda-8076-1d22a64fc595\") "
Sep 16 14:26:55.615688 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.615542    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47wbp\" (UniqueName: \"kubernetes.io/projected/253f179e-81c3-4dda-8076-1d22a64fc595-kube-api-access-47wbp\") pod \"253f179e-81c3-4dda-8076-1d22a64fc595\" (UID: \"253f179e-81c3-4dda-8076-1d22a64fc595\") "
Sep 16 14:26:55.615933 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.615912    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/253f179e-81c3-4dda-8076-1d22a64fc595-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "253f179e-81c3-4dda-8076-1d22a64fc595" (UID: "253f179e-81c3-4dda-8076-1d22a64fc595"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:26:55.615933 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.615915    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acccdabf-411a-4cc5-9c62-9add0560b775-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "acccdabf-411a-4cc5-9c62-9add0560b775" (UID: "acccdabf-411a-4cc5-9c62-9add0560b775"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:26:55.617742 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.617717    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acccdabf-411a-4cc5-9c62-9add0560b775-kube-api-access-l6pps" (OuterVolumeSpecName: "kube-api-access-l6pps") pod "acccdabf-411a-4cc5-9c62-9add0560b775" (UID: "acccdabf-411a-4cc5-9c62-9add0560b775"). InnerVolumeSpecName "kube-api-access-l6pps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:55.617865 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.617810    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/253f179e-81c3-4dda-8076-1d22a64fc595-kube-api-access-47wbp" (OuterVolumeSpecName: "kube-api-access-47wbp") pod "253f179e-81c3-4dda-8076-1d22a64fc595" (UID: "253f179e-81c3-4dda-8076-1d22a64fc595"). InnerVolumeSpecName "kube-api-access-47wbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:55.669039 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.669013    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"]
Sep 16 14:26:55.672869 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.672849    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c37e6e-nnvgr"]
Sep 16 14:26:55.716212 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.716187    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6pps\" (UniqueName: \"kubernetes.io/projected/acccdabf-411a-4cc5-9c62-9add0560b775-kube-api-access-l6pps\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:55.716314 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.716213    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acccdabf-411a-4cc5-9c62-9add0560b775-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:55.716314 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.716228    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/253f179e-81c3-4dda-8076-1d22a64fc595-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:55.716314 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.716240    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47wbp\" (UniqueName: \"kubernetes.io/projected/253f179e-81c3-4dda-8076-1d22a64fc595-kube-api-access-47wbp\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:26:55.860783 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.860691    2566 generic.go:356] "Generic (PLEG): container finished" podID="253f179e-81c3-4dda-8076-1d22a64fc595" containerID="1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5" exitCode=0
Sep 16 14:26:55.860783 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.860731    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"
Sep 16 14:26:55.860783 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.860774    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2" event={"ID":"253f179e-81c3-4dda-8076-1d22a64fc595","Type":"ContainerDied","Data":"1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5"}
Sep 16 14:26:55.861070 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.860827    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2" event={"ID":"253f179e-81c3-4dda-8076-1d22a64fc595","Type":"ContainerDied","Data":"dca3dcb1d24de89e957554e07438464c30d0839a8be4656c9ea03519726ac4d4"}
Sep 16 14:26:55.861070 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.860859    2566 scope.go:117] "RemoveContainer" containerID="1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5"
Sep 16 14:26:55.862288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.862267    2566 generic.go:356] "Generic (PLEG): container finished" podID="acccdabf-411a-4cc5-9c62-9add0560b775" containerID="335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9" exitCode=0
Sep 16 14:26:55.862390 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.862294    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7" event={"ID":"acccdabf-411a-4cc5-9c62-9add0560b775","Type":"ContainerDied","Data":"335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9"}
Sep 16 14:26:55.862390 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.862310    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"
Sep 16 14:26:55.862390 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.862343    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7" event={"ID":"acccdabf-411a-4cc5-9c62-9add0560b775","Type":"ContainerDied","Data":"426cfb0c4d406b8b2c73d84f78ac3684746a932c593e2bab3e2fcd10fb75582e"}
Sep 16 14:26:55.871174 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.871160    2566 scope.go:117] "RemoveContainer" containerID="1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5"
Sep 16 14:26:55.871494 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:26:55.871463    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5\": container with ID starting with 1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5 not found: ID does not exist" containerID="1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5"
Sep 16 14:26:55.871614 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.871495    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5"} err="failed to get container status \"1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5\": rpc error: code = NotFound desc = could not find container \"1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5\": container with ID starting with 1582a88f4d5a813f86ed01d290f8d88c3941039afe72fdc4d7f9f877dabb25f5 not found: ID does not exist"
Sep 16 14:26:55.871614 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.871513    2566 scope.go:117] "RemoveContainer" containerID="335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9"
Sep 16 14:26:55.879404 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.879387    2566 scope.go:117] "RemoveContainer" containerID="335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9"
Sep 16 14:26:55.879708 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:26:55.879690    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9\": container with ID starting with 335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9 not found: ID does not exist" containerID="335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9"
Sep 16 14:26:55.879764 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.879716    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9"} err="failed to get container status \"335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9\": rpc error: code = NotFound desc = could not find container \"335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9\": container with ID starting with 335202110b9f8367af5aed9886b74e2647ba08a4ef1ab9f1bba5b991e338eea9 not found: ID does not exist"
Sep 16 14:26:55.890190 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.890168    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"]
Sep 16 14:26:55.893476 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.893456    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c37e6e-68d5f4898-ptnd7"]
Sep 16 14:26:55.911561 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.911507    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"]
Sep 16 14:26:55.917237 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:55.917216    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c37e6e-6b6999d9c9-zlzn2"]
Sep 16 14:26:56.984586 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:56.984545    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="253f179e-81c3-4dda-8076-1d22a64fc595" path="/var/lib/kubelet/pods/253f179e-81c3-4dda-8076-1d22a64fc595/volumes"
Sep 16 14:26:56.985031 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:56.984896    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ea81d5e-14d4-4028-b90b-c2bc1f965dcb" path="/var/lib/kubelet/pods/8ea81d5e-14d4-4028-b90b-c2bc1f965dcb/volumes"
Sep 16 14:26:56.985201 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:26:56.985188    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acccdabf-411a-4cc5-9c62-9add0560b775" path="/var/lib/kubelet/pods/acccdabf-411a-4cc5-9c62-9add0560b775/volumes"
Sep 16 14:27:02.015884 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:02.015848    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"]
Sep 16 14:27:02.022086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:02.022063    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-286f09d7-fzhpr"]
Sep 16 14:27:02.985633 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:02.985594    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91179b4c-8576-4a08-b719-831530cb0f8b" path="/var/lib/kubelet/pods/91179b4c-8576-4a08-b719-831530cb0f8b/volumes"
Sep 16 14:27:04.289186 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.289129    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm" podUID="13d5673c-1465-4426-b6c7-230205396faa" containerName="main" containerID="cri-o://7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22" gracePeriod=45
Sep 16 14:27:04.558046 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.558023    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:27:04.592933 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.592903    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl8q9\" (UniqueName: \"kubernetes.io/projected/13d5673c-1465-4426-b6c7-230205396faa-kube-api-access-dl8q9\") pod \"13d5673c-1465-4426-b6c7-230205396faa\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") "
Sep 16 14:27:04.593108 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.592974    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-model-cache\") pod \"13d5673c-1465-4426-b6c7-230205396faa\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") "
Sep 16 14:27:04.593108 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.593021    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-home\") pod \"13d5673c-1465-4426-b6c7-230205396faa\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") "
Sep 16 14:27:04.593108 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.593079    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-dshm\") pod \"13d5673c-1465-4426-b6c7-230205396faa\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") "
Sep 16 14:27:04.593261 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.593108    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13d5673c-1465-4426-b6c7-230205396faa-tls-certs\") pod \"13d5673c-1465-4426-b6c7-230205396faa\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") "
Sep 16 14:27:04.593261 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.593145    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-tmp-dir\") pod \"13d5673c-1465-4426-b6c7-230205396faa\" (UID: \"13d5673c-1465-4426-b6c7-230205396faa\") "
Sep 16 14:27:04.593373 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.593236    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-model-cache" (OuterVolumeSpecName: "model-cache") pod "13d5673c-1465-4426-b6c7-230205396faa" (UID: "13d5673c-1465-4426-b6c7-230205396faa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:04.593373 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.593278    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-home" (OuterVolumeSpecName: "home") pod "13d5673c-1465-4426-b6c7-230205396faa" (UID: "13d5673c-1465-4426-b6c7-230205396faa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:04.593501 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.593478    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "13d5673c-1465-4426-b6c7-230205396faa" (UID: "13d5673c-1465-4426-b6c7-230205396faa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:04.593561 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.593498    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:04.593561 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.593525    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:04.595404 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.595376    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-dshm" (OuterVolumeSpecName: "dshm") pod "13d5673c-1465-4426-b6c7-230205396faa" (UID: "13d5673c-1465-4426-b6c7-230205396faa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:04.595500 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.595424    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13d5673c-1465-4426-b6c7-230205396faa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "13d5673c-1465-4426-b6c7-230205396faa" (UID: "13d5673c-1465-4426-b6c7-230205396faa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:27:04.595500 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.595466    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d5673c-1465-4426-b6c7-230205396faa-kube-api-access-dl8q9" (OuterVolumeSpecName: "kube-api-access-dl8q9") pod "13d5673c-1465-4426-b6c7-230205396faa" (UID: "13d5673c-1465-4426-b6c7-230205396faa"). InnerVolumeSpecName "kube-api-access-dl8q9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:04.694853 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.694821    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl8q9\" (UniqueName: \"kubernetes.io/projected/13d5673c-1465-4426-b6c7-230205396faa-kube-api-access-dl8q9\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:04.694853 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.694847    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:04.694853 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.694856    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13d5673c-1465-4426-b6c7-230205396faa-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:04.695079 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.694865    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13d5673c-1465-4426-b6c7-230205396faa-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:04.916036 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.915955    2566 generic.go:356] "Generic (PLEG): container finished" podID="13d5673c-1465-4426-b6c7-230205396faa" containerID="7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22" exitCode=0
Sep 16 14:27:04.916036 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.916003    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"
Sep 16 14:27:04.916036 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.916020    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm" event={"ID":"13d5673c-1465-4426-b6c7-230205396faa","Type":"ContainerDied","Data":"7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22"}
Sep 16 14:27:04.916283 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.916055    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm" event={"ID":"13d5673c-1465-4426-b6c7-230205396faa","Type":"ContainerDied","Data":"1199b0b3105173822e0c6dcb5f4352b21bdf7682abb0cfe13d0c1f95f6528121"}
Sep 16 14:27:04.916283 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.916073    2566 scope.go:117] "RemoveContainer" containerID="7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22"
Sep 16 14:27:04.925143 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.925123    2566 scope.go:117] "RemoveContainer" containerID="7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22"
Sep 16 14:27:04.925412 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:27:04.925393    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22\": container with ID starting with 7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22 not found: ID does not exist" containerID="7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22"
Sep 16 14:27:04.925464 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:04.925421    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22"} err="failed to get container status \"7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22\": rpc error: code = NotFound desc = could not find container \"7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22\": container with ID starting with 7c880d204b61e691a95ce6256d862fc8f6905767a223d93c0e30aeb45b6aab22 not found: ID does not exist"
Sep 16 14:27:05.026874 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:05.026840    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"]
Sep 16 14:27:05.040524 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:05.040496    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0adaea/test-model-0adaea-kserve-7b9b784669-rcznm"]
Sep 16 14:27:06.984454 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:06.984416    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13d5673c-1465-4426-b6c7-230205396faa" path="/var/lib/kubelet/pods/13d5673c-1465-4426-b6c7-230205396faa/volumes"
Sep 16 14:27:08.494769 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.494727    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"]
Sep 16 14:27:08.495239 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495134    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13d5673c-1465-4426-b6c7-230205396faa" containerName="main"
Sep 16 14:27:08.495239 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495148    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="13d5673c-1465-4426-b6c7-230205396faa" containerName="main"
Sep 16 14:27:08.495239 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495163    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="253f179e-81c3-4dda-8076-1d22a64fc595" containerName="payload-processing"
Sep 16 14:27:08.495239 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495168    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="253f179e-81c3-4dda-8076-1d22a64fc595" containerName="payload-processing"
Sep 16 14:27:08.495239 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495174    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91179b4c-8576-4a08-b719-831530cb0f8b" containerName="revoke-keys"
Sep 16 14:27:08.495239 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495181    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="91179b4c-8576-4a08-b719-831530cb0f8b" containerName="revoke-keys"
Sep 16 14:27:08.495239 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495196    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acccdabf-411a-4cc5-9c62-9add0560b775" containerName="payload-pre-processing"
Sep 16 14:27:08.495239 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495202    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="acccdabf-411a-4cc5-9c62-9add0560b775" containerName="payload-pre-processing"
Sep 16 14:27:08.495639 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495267    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="253f179e-81c3-4dda-8076-1d22a64fc595" containerName="payload-processing"
Sep 16 14:27:08.495639 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495274    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="acccdabf-411a-4cc5-9c62-9add0560b775" containerName="payload-pre-processing"
Sep 16 14:27:08.495639 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495280    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="91179b4c-8576-4a08-b719-831530cb0f8b" containerName="revoke-keys"
Sep 16 14:27:08.495639 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.495287    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="13d5673c-1465-4426-b6c7-230205396faa" containerName="main"
Sep 16 14:27:08.499475 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.499451    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"
Sep 16 14:27:08.502216 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.502186    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:27:08.502342 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.502324    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:27:08.522936 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.522909    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"]
Sep 16 14:27:08.528131 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.528106    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f919b613-c1cb-4f2b-9fff-e9086f4c9f17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq\" (UID: \"f919b613-c1cb-4f2b-9fff-e9086f4c9f17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"
Sep 16 14:27:08.629521 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.629487    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f919b613-c1cb-4f2b-9fff-e9086f4c9f17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq\" (UID: \"f919b613-c1cb-4f2b-9fff-e9086f4c9f17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"
Sep 16 14:27:08.630107 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.630087    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f919b613-c1cb-4f2b-9fff-e9086f4c9f17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq\" (UID: \"f919b613-c1cb-4f2b-9fff-e9086f4c9f17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"
Sep 16 14:27:08.809291 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:08.809212    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"
Sep 16 14:27:09.087329 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.087299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"]
Sep 16 14:27:09.089282 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:27:09.089250    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf919b613_c1cb_4f2b_9fff_e9086f4c9f17.slice/crio-49aae022568d31893f32e747dbd817905d269100b4ccca7070af00da1c8f9bfd WatchSource:0}: Error finding container 49aae022568d31893f32e747dbd817905d269100b4ccca7070af00da1c8f9bfd: Status 404 returned error can't find the container with id 49aae022568d31893f32e747dbd817905d269100b4ccca7070af00da1c8f9bfd
Sep 16 14:27:09.736297 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.736261    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"]
Sep 16 14:27:09.739515 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.739495    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:09.743272 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.743255    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-beaf471d\""
Sep 16 14:27:09.765974 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.765949    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"]
Sep 16 14:27:09.838963 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.838926    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d98d0725-7331-4a25-8053-ba621001438c-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns\" (UID: \"d98d0725-7331-4a25-8053-ba621001438c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:09.839086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.838971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gclnb\" (UniqueName: \"kubernetes.io/projected/d98d0725-7331-4a25-8053-ba621001438c-kube-api-access-gclnb\") pod \"payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns\" (UID: \"d98d0725-7331-4a25-8053-ba621001438c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:09.939665 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.939630    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d98d0725-7331-4a25-8053-ba621001438c-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns\" (UID: \"d98d0725-7331-4a25-8053-ba621001438c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:09.939859 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.939672    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gclnb\" (UniqueName: \"kubernetes.io/projected/d98d0725-7331-4a25-8053-ba621001438c-kube-api-access-gclnb\") pod \"payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns\" (UID: \"d98d0725-7331-4a25-8053-ba621001438c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:09.940516 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.940490    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d98d0725-7331-4a25-8053-ba621001438c-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns\" (UID: \"d98d0725-7331-4a25-8053-ba621001438c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:09.941226 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.941197    2566 generic.go:356] "Generic (PLEG): container finished" podID="f919b613-c1cb-4f2b-9fff-e9086f4c9f17" containerID="02bd076f37e52b2d3d4e8ae915ab1002c669aaaaf2e61ae06b800450f3473b61" exitCode=0
Sep 16 14:27:09.941314 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.941274    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq" event={"ID":"f919b613-c1cb-4f2b-9fff-e9086f4c9f17","Type":"ContainerDied","Data":"02bd076f37e52b2d3d4e8ae915ab1002c669aaaaf2e61ae06b800450f3473b61"}
Sep 16 14:27:09.941356 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.941319    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq" event={"ID":"f919b613-c1cb-4f2b-9fff-e9086f4c9f17","Type":"ContainerStarted","Data":"49aae022568d31893f32e747dbd817905d269100b4ccca7070af00da1c8f9bfd"}
Sep 16 14:27:09.954022 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:09.954001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gclnb\" (UniqueName: \"kubernetes.io/projected/d98d0725-7331-4a25-8053-ba621001438c-kube-api-access-gclnb\") pod \"payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns\" (UID: \"d98d0725-7331-4a25-8053-ba621001438c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:10.049673 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:10.049604    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:10.343680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:10.343656    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"]
Sep 16 14:27:10.346093 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:27:10.346068    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98d0725_7331_4a25_8053_ba621001438c.slice/crio-024b33f17f2aedd2a5d9e23d8931737dbea36e25bf63aaba89ad2793c7fb0a1d WatchSource:0}: Error finding container 024b33f17f2aedd2a5d9e23d8931737dbea36e25bf63aaba89ad2793c7fb0a1d: Status 404 returned error can't find the container with id 024b33f17f2aedd2a5d9e23d8931737dbea36e25bf63aaba89ad2793c7fb0a1d
Sep 16 14:27:10.947227 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:10.947187    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns" event={"ID":"d98d0725-7331-4a25-8053-ba621001438c","Type":"ContainerStarted","Data":"ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0"}
Sep 16 14:27:10.947674 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:10.947245    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns" event={"ID":"d98d0725-7331-4a25-8053-ba621001438c","Type":"ContainerStarted","Data":"024b33f17f2aedd2a5d9e23d8931737dbea36e25bf63aaba89ad2793c7fb0a1d"}
Sep 16 14:27:10.983527 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:10.983469    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns" podStartSLOduration=1.9834498489999999 podStartE2EDuration="1.983449849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:10.980627516 +0000 UTC m=+1150.457859602" watchObservedRunningTime="2026-09-16 14:27:10.983449849 +0000 UTC m=+1150.460681925"
Sep 16 14:27:11.077405 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:11.077381    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"
Sep 16 14:27:11.149541 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:11.149511    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f919b613-c1cb-4f2b-9fff-e9086f4c9f17-maas-api-service-ca\") pod \"f919b613-c1cb-4f2b-9fff-e9086f4c9f17\" (UID: \"f919b613-c1cb-4f2b-9fff-e9086f4c9f17\") "
Sep 16 14:27:11.149967 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:11.149941    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f919b613-c1cb-4f2b-9fff-e9086f4c9f17-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f919b613-c1cb-4f2b-9fff-e9086f4c9f17" (UID: "f919b613-c1cb-4f2b-9fff-e9086f4c9f17"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:11.250821 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:11.250743    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f919b613-c1cb-4f2b-9fff-e9086f4c9f17-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:11.952836 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:11.952810    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"
Sep 16 14:27:11.953280 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:11.952851    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq" event={"ID":"f919b613-c1cb-4f2b-9fff-e9086f4c9f17","Type":"ContainerDied","Data":"49aae022568d31893f32e747dbd817905d269100b4ccca7070af00da1c8f9bfd"}
Sep 16 14:27:11.953280 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:11.952927    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49aae022568d31893f32e747dbd817905d269100b4ccca7070af00da1c8f9bfd"
Sep 16 14:27:11.953280 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:11.953200    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:19.748036 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:19.748001    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"]
Sep 16 14:27:19.748653 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:19.748219    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv" podUID="6ff9c51c-e043-4081-9cbc-43bf3fc73c8e" containerName="payload-processing" containerID="cri-o://bc626399a7dbaa8a8126dd18b23996d83471d68d313ae7c29375dfdef27f1c40" gracePeriod=30
Sep 16 14:27:19.763667 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:19.763556    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"]
Sep 16 14:27:19.763878 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:19.763857    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz" podUID="0591d7d3-b995-4ae3-9664-22d5fc8aad3a" containerName="payload-pre-processing" containerID="cri-o://8ce089e61b3a845c4ddad0f8ad09ba1ed36cf248751bda828ccdbaaa32e49ff1" gracePeriod=30
Sep 16 14:27:19.996867 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:19.996838    2566 generic.go:356] "Generic (PLEG): container finished" podID="0591d7d3-b995-4ae3-9664-22d5fc8aad3a" containerID="8ce089e61b3a845c4ddad0f8ad09ba1ed36cf248751bda828ccdbaaa32e49ff1" exitCode=0
Sep 16 14:27:19.997008 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:19.996906    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz" event={"ID":"0591d7d3-b995-4ae3-9664-22d5fc8aad3a","Type":"ContainerDied","Data":"8ce089e61b3a845c4ddad0f8ad09ba1ed36cf248751bda828ccdbaaa32e49ff1"}
Sep 16 14:27:19.998579 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:19.998503    2566 generic.go:356] "Generic (PLEG): container finished" podID="6ff9c51c-e043-4081-9cbc-43bf3fc73c8e" containerID="bc626399a7dbaa8a8126dd18b23996d83471d68d313ae7c29375dfdef27f1c40" exitCode=0
Sep 16 14:27:19.998579 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:19.998545    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv" event={"ID":"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e","Type":"ContainerDied","Data":"bc626399a7dbaa8a8126dd18b23996d83471d68d313ae7c29375dfdef27f1c40"}
Sep 16 14:27:20.040475 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.040445    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:27:20.042697 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.042681    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:27:20.125511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.125481    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnnhs\" (UniqueName: \"kubernetes.io/projected/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-kube-api-access-mnnhs\") pod \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\" (UID: \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\") "
Sep 16 14:27:20.125810 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.125528    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-plugins-config-volume\") pod \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\" (UID: \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\") "
Sep 16 14:27:20.125810 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.125638    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-plugins-config-volume\") pod \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\" (UID: \"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e\") "
Sep 16 14:27:20.125810 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.125669    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxpnv\" (UniqueName: \"kubernetes.io/projected/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-kube-api-access-dxpnv\") pod \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\" (UID: \"0591d7d3-b995-4ae3-9664-22d5fc8aad3a\") "
Sep 16 14:27:20.126505 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.126475    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0591d7d3-b995-4ae3-9664-22d5fc8aad3a" (UID: "0591d7d3-b995-4ae3-9664-22d5fc8aad3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:20.126776 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.126756    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ff9c51c-e043-4081-9cbc-43bf3fc73c8e" (UID: "6ff9c51c-e043-4081-9cbc-43bf3fc73c8e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:20.128408 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.128389    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-kube-api-access-mnnhs" (OuterVolumeSpecName: "kube-api-access-mnnhs") pod "6ff9c51c-e043-4081-9cbc-43bf3fc73c8e" (UID: "6ff9c51c-e043-4081-9cbc-43bf3fc73c8e"). InnerVolumeSpecName "kube-api-access-mnnhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:20.128516 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.128429    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-kube-api-access-dxpnv" (OuterVolumeSpecName: "kube-api-access-dxpnv") pod "0591d7d3-b995-4ae3-9664-22d5fc8aad3a" (UID: "0591d7d3-b995-4ae3-9664-22d5fc8aad3a"). InnerVolumeSpecName "kube-api-access-dxpnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:20.128818 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.128798    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"]
Sep 16 14:27:20.133320 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.133298    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0adaea-4jkzq"]
Sep 16 14:27:20.227286 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.227259    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnnhs\" (UniqueName: \"kubernetes.io/projected/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-kube-api-access-mnnhs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.227286 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.227283    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.227286 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.227293    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.227511 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.227302    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxpnv\" (UniqueName: \"kubernetes.io/projected/0591d7d3-b995-4ae3-9664-22d5fc8aad3a-kube-api-access-dxpnv\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.988592 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:20.985838    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f919b613-c1cb-4f2b-9fff-e9086f4c9f17" path="/var/lib/kubelet/pods/f919b613-c1cb-4f2b-9fff-e9086f4c9f17/volumes"
Sep 16 14:27:21.005443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.005419    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"
Sep 16 14:27:21.005556 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.005442    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz" event={"ID":"0591d7d3-b995-4ae3-9664-22d5fc8aad3a","Type":"ContainerDied","Data":"c24a08e6d22f6b98633ae4f8252e62864f47cd2d5dbec7d23c4af8d04f9ab83c"}
Sep 16 14:27:21.005556 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.005508    2566 scope.go:117] "RemoveContainer" containerID="8ce089e61b3a845c4ddad0f8ad09ba1ed36cf248751bda828ccdbaaa32e49ff1"
Sep 16 14:27:21.006904 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.006847    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"
Sep 16 14:27:21.006904 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.006849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv" event={"ID":"6ff9c51c-e043-4081-9cbc-43bf3fc73c8e","Type":"ContainerDied","Data":"f4202f35f16a02cad06950d52db4066b944cf16fa78faa6e0086884c5d8b3b8f"}
Sep 16 14:27:21.014317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.014301    2566 scope.go:117] "RemoveContainer" containerID="bc626399a7dbaa8a8126dd18b23996d83471d68d313ae7c29375dfdef27f1c40"
Sep 16 14:27:21.032450 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032430    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"]
Sep 16 14:27:21.032795 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032782    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f919b613-c1cb-4f2b-9fff-e9086f4c9f17" containerName="revoke-keys"
Sep 16 14:27:21.032841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032797    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f919b613-c1cb-4f2b-9fff-e9086f4c9f17" containerName="revoke-keys"
Sep 16 14:27:21.032841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032806    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0591d7d3-b995-4ae3-9664-22d5fc8aad3a" containerName="payload-pre-processing"
Sep 16 14:27:21.032841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032812    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0591d7d3-b995-4ae3-9664-22d5fc8aad3a" containerName="payload-pre-processing"
Sep 16 14:27:21.032841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032820    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff9c51c-e043-4081-9cbc-43bf3fc73c8e" containerName="payload-processing"
Sep 16 14:27:21.032841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032825    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff9c51c-e043-4081-9cbc-43bf3fc73c8e" containerName="payload-processing"
Sep 16 14:27:21.032981 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032880    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0591d7d3-b995-4ae3-9664-22d5fc8aad3a" containerName="payload-pre-processing"
Sep 16 14:27:21.032981 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032887    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f919b613-c1cb-4f2b-9fff-e9086f4c9f17" containerName="revoke-keys"
Sep 16 14:27:21.032981 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.032894    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff9c51c-e043-4081-9cbc-43bf3fc73c8e" containerName="payload-processing"
Sep 16 14:27:21.037430 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.037414    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.044190 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.044168    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"distinct-2-w0-11636c-kserve-self-signed-certs\""
Sep 16 14:27:21.050346 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.050324    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"]
Sep 16 14:27:21.059444 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.059423    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"]
Sep 16 14:27:21.075604 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.075559    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0adaea-85bb45d4c9-k27dv"]
Sep 16 14:27:21.111554 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.111534    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"]
Sep 16 14:27:21.113124 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.113106    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0adaea-c94c65f7d-42znz"]
Sep 16 14:27:21.134948 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.134926    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-dshm\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.135028 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.134966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-home\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.135028 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.135012    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19149d56-37f9-48af-842a-a7c3e00e8fa8-tls-certs\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.135096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.135042    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-tmp-dir\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.135096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.135085    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m94r\" (UniqueName: \"kubernetes.io/projected/19149d56-37f9-48af-842a-a7c3e00e8fa8-kube-api-access-5m94r\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.135168 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.135128    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-model-cache\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.235915 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.235884    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-home\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.235915 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.235931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19149d56-37f9-48af-842a-a7c3e00e8fa8-tls-certs\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.236141 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.236059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-tmp-dir\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.236141 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.236132    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m94r\" (UniqueName: \"kubernetes.io/projected/19149d56-37f9-48af-842a-a7c3e00e8fa8-kube-api-access-5m94r\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.236229 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.236197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-model-cache\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.236285 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.236265    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-home\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.236383 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.236357    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-dshm\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.236468 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.236388    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-tmp-dir\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.236468 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.236431    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-model-cache\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.238383 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.238356    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-dshm\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.238703 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.238687    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19149d56-37f9-48af-842a-a7c3e00e8fa8-tls-certs\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.251923 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.251902    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m94r\" (UniqueName: \"kubernetes.io/projected/19149d56-37f9-48af-842a-a7c3e00e8fa8-kube-api-access-5m94r\") pod \"distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") " pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.319265 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.319182    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:27:21.346141 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.346107    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:21.665776 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:21.665682    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"]
Sep 16 14:27:21.670060 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:27:21.670028    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19149d56_37f9_48af_842a_a7c3e00e8fa8.slice/crio-54934de1afee0963876e8104715437e658ccc461bf4755f626be027afd323eba WatchSource:0}: Error finding container 54934de1afee0963876e8104715437e658ccc461bf4755f626be027afd323eba: Status 404 returned error can't find the container with id 54934de1afee0963876e8104715437e658ccc461bf4755f626be027afd323eba
Sep 16 14:27:22.015749 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:22.015706    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn" event={"ID":"19149d56-37f9-48af-842a-a7c3e00e8fa8","Type":"ContainerStarted","Data":"8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b"}
Sep 16 14:27:22.015749 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:22.015755    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn" event={"ID":"19149d56-37f9-48af-842a-a7c3e00e8fa8","Type":"ContainerStarted","Data":"54934de1afee0963876e8104715437e658ccc461bf4755f626be027afd323eba"}
Sep 16 14:27:22.960372 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:22.960295    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:22.984714 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:22.984686    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0591d7d3-b995-4ae3-9664-22d5fc8aad3a" path="/var/lib/kubelet/pods/0591d7d3-b995-4ae3-9664-22d5fc8aad3a/volumes"
Sep 16 14:27:22.985053 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:22.985037    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ff9c51c-e043-4081-9cbc-43bf3fc73c8e" path="/var/lib/kubelet/pods/6ff9c51c-e043-4081-9cbc-43bf3fc73c8e/volumes"
Sep 16 14:27:22.996283 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:22.996234    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn" podStartSLOduration=2.996221448 podStartE2EDuration="2.996221448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:22.067882731 +0000 UTC m=+1161.545114805" watchObservedRunningTime="2026-09-16 14:27:22.996221448 +0000 UTC m=+1162.473453513"
Sep 16 14:27:25.765690 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:25.765653    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"]
Sep 16 14:27:25.770439 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:25.770417    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"
Sep 16 14:27:25.776263 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:25.776239    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:27:25.776412 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:25.776239    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:27:25.794820 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:25.794795    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"]
Sep 16 14:27:25.890204 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:25.890169    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-beaf471d-6228f\" (UID: \"f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"
Sep 16 14:27:25.991316 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:25.991283    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-beaf471d-6228f\" (UID: \"f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"
Sep 16 14:27:25.991886 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:25.991870    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-beaf471d-6228f\" (UID: \"f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"
Sep 16 14:27:26.080734 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:26.080659    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"
Sep 16 14:27:26.355722 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:26.355655    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"]
Sep 16 14:27:26.358845 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:27:26.358807    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf08bfb1f_eaf0_4c77_bdfa_dfabf9915b5b.slice/crio-0ca7825e88b8425c7f166cffc188d0add3a17157256515f667c9791e71c54723 WatchSource:0}: Error finding container 0ca7825e88b8425c7f166cffc188d0add3a17157256515f667c9791e71c54723: Status 404 returned error can't find the container with id 0ca7825e88b8425c7f166cffc188d0add3a17157256515f667c9791e71c54723
Sep 16 14:27:27.049646 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:27.049611    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f" event={"ID":"f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b","Type":"ContainerStarted","Data":"2b155bd380d06af6c764a1c89ccd7d59e4e0a9630ffd87a8f205f9a1f1d5e194"}
Sep 16 14:27:27.050013 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:27.049660    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f" event={"ID":"f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b","Type":"ContainerStarted","Data":"0ca7825e88b8425c7f166cffc188d0add3a17157256515f667c9791e71c54723"}
Sep 16 14:27:27.090510 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:27.090466    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f" podStartSLOduration=2.090452386 podStartE2EDuration="2.090452386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:27.086972995 +0000 UTC m=+1166.564205071" watchObservedRunningTime="2026-09-16 14:27:27.090452386 +0000 UTC m=+1166.567684460"
Sep 16 14:27:28.055378 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:28.055344    2566 generic.go:356] "Generic (PLEG): container finished" podID="f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b" containerID="2b155bd380d06af6c764a1c89ccd7d59e4e0a9630ffd87a8f205f9a1f1d5e194" exitCode=0
Sep 16 14:27:28.055781 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:28.055435    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f" event={"ID":"f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b","Type":"ContainerDied","Data":"2b155bd380d06af6c764a1c89ccd7d59e4e0a9630ffd87a8f205f9a1f1d5e194"}
Sep 16 14:27:29.189257 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:29.189232    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"
Sep 16 14:27:29.321553 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:29.321462    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b-maas-api-service-ca\") pod \"f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b\" (UID: \"f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b\") "
Sep 16 14:27:29.321857 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:29.321835    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b" (UID: "f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:29.422169 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:29.422135    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:30.065508 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:30.065470    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f" event={"ID":"f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b","Type":"ContainerDied","Data":"0ca7825e88b8425c7f166cffc188d0add3a17157256515f667c9791e71c54723"}
Sep 16 14:27:30.065687 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:30.065527    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ca7825e88b8425c7f166cffc188d0add3a17157256515f667c9791e71c54723"
Sep 16 14:27:30.065687 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:30.065492    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"
Sep 16 14:27:31.346922 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:31.346893    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:31.347395 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:31.346934    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:31.359376 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:31.359346    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:32.086072 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:32.086042    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:27:38.512425 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:38.512392    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"]
Sep 16 14:27:38.512851 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:38.512651    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns" podUID="d98d0725-7331-4a25-8053-ba621001438c" containerName="payload-pre-processing" containerID="cri-o://ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0" gracePeriod=30
Sep 16 14:27:38.756809 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:38.756781    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:38.908406 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:38.908328    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gclnb\" (UniqueName: \"kubernetes.io/projected/d98d0725-7331-4a25-8053-ba621001438c-kube-api-access-gclnb\") pod \"d98d0725-7331-4a25-8053-ba621001438c\" (UID: \"d98d0725-7331-4a25-8053-ba621001438c\") "
Sep 16 14:27:38.908556 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:38.908452    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d98d0725-7331-4a25-8053-ba621001438c-plugins-config-volume\") pod \"d98d0725-7331-4a25-8053-ba621001438c\" (UID: \"d98d0725-7331-4a25-8053-ba621001438c\") "
Sep 16 14:27:38.908833 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:38.908809    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d98d0725-7331-4a25-8053-ba621001438c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d98d0725-7331-4a25-8053-ba621001438c" (UID: "d98d0725-7331-4a25-8053-ba621001438c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:38.910611 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:38.910593    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d98d0725-7331-4a25-8053-ba621001438c-kube-api-access-gclnb" (OuterVolumeSpecName: "kube-api-access-gclnb") pod "d98d0725-7331-4a25-8053-ba621001438c" (UID: "d98d0725-7331-4a25-8053-ba621001438c"). InnerVolumeSpecName "kube-api-access-gclnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:39.009854 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.009819    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d98d0725-7331-4a25-8053-ba621001438c-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:39.009854 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.009847    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gclnb\" (UniqueName: \"kubernetes.io/projected/d98d0725-7331-4a25-8053-ba621001438c-kube-api-access-gclnb\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:27:39.108624 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.108589    2566 generic.go:356] "Generic (PLEG): container finished" podID="d98d0725-7331-4a25-8053-ba621001438c" containerID="ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0" exitCode=0
Sep 16 14:27:39.108807 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.108633    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"
Sep 16 14:27:39.108807 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.108682    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns" event={"ID":"d98d0725-7331-4a25-8053-ba621001438c","Type":"ContainerDied","Data":"ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0"}
Sep 16 14:27:39.108807 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.108733    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns" event={"ID":"d98d0725-7331-4a25-8053-ba621001438c","Type":"ContainerDied","Data":"024b33f17f2aedd2a5d9e23d8931737dbea36e25bf63aaba89ad2793c7fb0a1d"}
Sep 16 14:27:39.108807 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.108754    2566 scope.go:117] "RemoveContainer" containerID="ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0"
Sep 16 14:27:39.117241 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.117222    2566 scope.go:117] "RemoveContainer" containerID="ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0"
Sep 16 14:27:39.117548 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:27:39.117520    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0\": container with ID starting with ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0 not found: ID does not exist" containerID="ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0"
Sep 16 14:27:39.117703 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.117554    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0"} err="failed to get container status \"ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0\": rpc error: code = NotFound desc = could not find container \"ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0\": container with ID starting with ec0c1761d747b06c479669b5cd6636d93035e29554d41d9e3e6d066240d7b6c0 not found: ID does not exist"
Sep 16 14:27:39.146146 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.146117    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"]
Sep 16 14:27:39.158846 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.158781    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-beaf471d-6228f"]
Sep 16 14:27:39.160258 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.160229    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"]
Sep 16 14:27:39.186330 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:39.186296    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-beaf471d-849bfc8966-grbns"]
Sep 16 14:27:40.683665 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.683635    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"]
Sep 16 14:27:40.684031 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.683994    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d98d0725-7331-4a25-8053-ba621001438c" containerName="payload-pre-processing"
Sep 16 14:27:40.684031 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.684005    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d98d0725-7331-4a25-8053-ba621001438c" containerName="payload-pre-processing"
Sep 16 14:27:40.684031 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.684026    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b" containerName="revoke-keys"
Sep 16 14:27:40.684031 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.684032    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b" containerName="revoke-keys"
Sep 16 14:27:40.684171 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.684087    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b" containerName="revoke-keys"
Sep 16 14:27:40.684171 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.684102    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d98d0725-7331-4a25-8053-ba621001438c" containerName="payload-pre-processing"
Sep 16 14:27:40.688264 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.688246    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:40.691456 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.691437    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-c0974e-dockercfg-9s6ll\""
Sep 16 14:27:40.691789 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.691769    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-c0974e\""
Sep 16 14:27:40.702577 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.702539    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"]
Sep 16 14:27:40.789366 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.789330    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"]
Sep 16 14:27:40.792582 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.792551    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:40.811737 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.811711    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"]
Sep 16 14:27:40.828066 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.828042    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbzgb\" (UniqueName: \"kubernetes.io/projected/1c3e1058-31f0-47e6-ba05-191fc0934793-kube-api-access-bbzgb\") pod \"payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4\" (UID: \"1c3e1058-31f0-47e6-ba05-191fc0934793\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:40.828175 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.828079    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3e1058-31f0-47e6-ba05-191fc0934793-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4\" (UID: \"1c3e1058-31f0-47e6-ba05-191fc0934793\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:40.928579 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.928534    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvg7h\" (UniqueName: \"kubernetes.io/projected/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-kube-api-access-hvg7h\") pod \"payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph\" (UID: \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:40.928734 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.928685    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph\" (UID: \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:40.928734 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.928725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbzgb\" (UniqueName: \"kubernetes.io/projected/1c3e1058-31f0-47e6-ba05-191fc0934793-kube-api-access-bbzgb\") pod \"payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4\" (UID: \"1c3e1058-31f0-47e6-ba05-191fc0934793\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:40.928817 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.928751    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3e1058-31f0-47e6-ba05-191fc0934793-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4\" (UID: \"1c3e1058-31f0-47e6-ba05-191fc0934793\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:40.929340 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.929323    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3e1058-31f0-47e6-ba05-191fc0934793-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4\" (UID: \"1c3e1058-31f0-47e6-ba05-191fc0934793\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:40.937681 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.937625    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbzgb\" (UniqueName: \"kubernetes.io/projected/1c3e1058-31f0-47e6-ba05-191fc0934793-kube-api-access-bbzgb\") pod \"payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4\" (UID: \"1c3e1058-31f0-47e6-ba05-191fc0934793\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:40.984320 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.984290    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d98d0725-7331-4a25-8053-ba621001438c" path="/var/lib/kubelet/pods/d98d0725-7331-4a25-8053-ba621001438c/volumes"
Sep 16 14:27:40.984644 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.984630    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b" path="/var/lib/kubelet/pods/f08bfb1f-eaf0-4c77-bdfa-dfabf9915b5b/volumes"
Sep 16 14:27:40.998395 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:40.998369    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:41.029761 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:41.029736    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph\" (UID: \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:41.029880 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:41.029805    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvg7h\" (UniqueName: \"kubernetes.io/projected/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-kube-api-access-hvg7h\") pod \"payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph\" (UID: \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:41.030478 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:41.030462    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph\" (UID: \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:41.042720 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:41.042696    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvg7h\" (UniqueName: \"kubernetes.io/projected/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-kube-api-access-hvg7h\") pod \"payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph\" (UID: \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:41.102965 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:41.102929    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:41.275013 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:41.274988    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"]
Sep 16 14:27:41.276069 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:27:41.276044    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3e1058_31f0_47e6_ba05_191fc0934793.slice/crio-67e63a51a0bca4ff7cc473358cf047436988f51eea20402b49cdbdd7779eaa38 WatchSource:0}: Error finding container 67e63a51a0bca4ff7cc473358cf047436988f51eea20402b49cdbdd7779eaa38: Status 404 returned error can't find the container with id 67e63a51a0bca4ff7cc473358cf047436988f51eea20402b49cdbdd7779eaa38
Sep 16 14:27:41.375297 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:41.375268    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"]
Sep 16 14:27:41.376644 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:27:41.376621    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe5aa8e_00d7_4c06_9367_f6d6b70e58ed.slice/crio-a23e77cc2d4323758156ce578ac0c4a995c36938b8ce87a8bc22604c31025746 WatchSource:0}: Error finding container a23e77cc2d4323758156ce578ac0c4a995c36938b8ce87a8bc22604c31025746: Status 404 returned error can't find the container with id a23e77cc2d4323758156ce578ac0c4a995c36938b8ce87a8bc22604c31025746
Sep 16 14:27:42.128725 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:42.128530    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph" event={"ID":"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed","Type":"ContainerStarted","Data":"ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb"}
Sep 16 14:27:42.128725 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:42.128655    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph" event={"ID":"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed","Type":"ContainerStarted","Data":"a23e77cc2d4323758156ce578ac0c4a995c36938b8ce87a8bc22604c31025746"}
Sep 16 14:27:42.128725 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:42.128690    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:42.130438 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:42.130405    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4" event={"ID":"1c3e1058-31f0-47e6-ba05-191fc0934793","Type":"ContainerStarted","Data":"8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da"}
Sep 16 14:27:42.130560 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:42.130454    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4" event={"ID":"1c3e1058-31f0-47e6-ba05-191fc0934793","Type":"ContainerStarted","Data":"67e63a51a0bca4ff7cc473358cf047436988f51eea20402b49cdbdd7779eaa38"}
Sep 16 14:27:42.130560 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:42.130505    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:42.163917 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:42.163862    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph" podStartSLOduration=2.16384647 podStartE2EDuration="2.16384647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:42.161657157 +0000 UTC m=+1181.638889253" watchObservedRunningTime="2026-09-16 14:27:42.16384647 +0000 UTC m=+1181.641078542"
Sep 16 14:27:42.222744 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:42.222689    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4" podStartSLOduration=2.222673138 podStartE2EDuration="2.222673138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:42.220171517 +0000 UTC m=+1181.697403611" watchObservedRunningTime="2026-09-16 14:27:42.222673138 +0000 UTC m=+1181.699905213"
Sep 16 14:27:45.436930 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.436898    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"]
Sep 16 14:27:45.440620 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.440603    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:27:45.443637 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.443615    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5f53f6be\""
Sep 16 14:27:45.443755 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.443704    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5f53f6be-dockercfg-djq6m\""
Sep 16 14:27:45.451011 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.450987    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"]
Sep 16 14:27:45.468087 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.468064    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5btsk\" (UniqueName: \"kubernetes.io/projected/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-kube-api-access-5btsk\") pod \"payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp\" (UID: \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:27:45.468183 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.468136    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp\" (UID: \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:27:45.569279 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.569247    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp\" (UID: \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:27:45.569443 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.569299    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5btsk\" (UniqueName: \"kubernetes.io/projected/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-kube-api-access-5btsk\") pod \"payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp\" (UID: \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:27:45.569963 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.569944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp\" (UID: \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:27:45.578813 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.578782    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5btsk\" (UniqueName: \"kubernetes.io/projected/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-kube-api-access-5btsk\") pod \"payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp\" (UID: \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:27:45.751269 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:45.751233    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:27:46.044701 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:46.044675    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"]
Sep 16 14:27:46.047193 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:27:46.047151    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfdeeff_f76c_473c_8bd3_0558fb2739e3.slice/crio-c2a588a809f01283077816247cfdc5856adc0fc98a03c58c6d3dd2676a9bcfa8 WatchSource:0}: Error finding container c2a588a809f01283077816247cfdc5856adc0fc98a03c58c6d3dd2676a9bcfa8: Status 404 returned error can't find the container with id c2a588a809f01283077816247cfdc5856adc0fc98a03c58c6d3dd2676a9bcfa8
Sep 16 14:27:46.153644 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:46.153609    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp" event={"ID":"7bfdeeff-f76c-473c-8bd3-0558fb2739e3","Type":"ContainerStarted","Data":"72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4"}
Sep 16 14:27:46.153817 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:46.153680    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:27:46.153817 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:46.153702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp" event={"ID":"7bfdeeff-f76c-473c-8bd3-0558fb2739e3","Type":"ContainerStarted","Data":"c2a588a809f01283077816247cfdc5856adc0fc98a03c58c6d3dd2676a9bcfa8"}
Sep 16 14:27:46.254392 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:46.254343    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp" podStartSLOduration=1.25432779 podStartE2EDuration="1.25432779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:46.253344331 +0000 UTC m=+1185.730576417" watchObservedRunningTime="2026-09-16 14:27:46.25432779 +0000 UTC m=+1185.731559895"
Sep 16 14:27:53.138742 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:53.138708    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:27:53.139124 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:53.138869    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:27:57.161055 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:27:57.161020    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:28:00.470283 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:00.470250    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:28:08.984777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:08.984744    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"]
Sep 16 14:28:08.989710 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:08.989689    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"
Sep 16 14:28:08.992157 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:08.992136    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:28:08.992278 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:08.992203    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:28:08.997967 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:08.997946    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"]
Sep 16 14:28:09.080032 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:09.079992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea32e654-51eb-42a0-a3ae-70119a46cec8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf\" (UID: \"ea32e654-51eb-42a0-a3ae-70119a46cec8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"
Sep 16 14:28:09.181201 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:09.181173    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea32e654-51eb-42a0-a3ae-70119a46cec8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf\" (UID: \"ea32e654-51eb-42a0-a3ae-70119a46cec8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"
Sep 16 14:28:09.181807 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:09.181790    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea32e654-51eb-42a0-a3ae-70119a46cec8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf\" (UID: \"ea32e654-51eb-42a0-a3ae-70119a46cec8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"
Sep 16 14:28:09.301840 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:09.301759    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"
Sep 16 14:28:09.574925 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:09.574903    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"]
Sep 16 14:28:09.577073 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:28:09.577044    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea32e654_51eb_42a0_a3ae_70119a46cec8.slice/crio-bff74c1238222d19f631ac6943db24df87f1a817614d6780b61dbe99312c40b9 WatchSource:0}: Error finding container bff74c1238222d19f631ac6943db24df87f1a817614d6780b61dbe99312c40b9: Status 404 returned error can't find the container with id bff74c1238222d19f631ac6943db24df87f1a817614d6780b61dbe99312c40b9
Sep 16 14:28:10.278950 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:10.278906    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf" event={"ID":"ea32e654-51eb-42a0-a3ae-70119a46cec8","Type":"ContainerStarted","Data":"6de3e1915d91626753c6ed1391484369da208523d20372efafbe555ce417caeb"}
Sep 16 14:28:10.279429 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:10.279015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf" event={"ID":"ea32e654-51eb-42a0-a3ae-70119a46cec8","Type":"ContainerStarted","Data":"bff74c1238222d19f631ac6943db24df87f1a817614d6780b61dbe99312c40b9"}
Sep 16 14:28:10.302580 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:10.302522    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf" podStartSLOduration=2.302507334 podStartE2EDuration="2.302507334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:10.30016058 +0000 UTC m=+1209.777392674" watchObservedRunningTime="2026-09-16 14:28:10.302507334 +0000 UTC m=+1209.779739435"
Sep 16 14:28:11.284275 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:11.284239    2566 generic.go:356] "Generic (PLEG): container finished" podID="ea32e654-51eb-42a0-a3ae-70119a46cec8" containerID="6de3e1915d91626753c6ed1391484369da208523d20372efafbe555ce417caeb" exitCode=0
Sep 16 14:28:11.284688 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:11.284326    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf" event={"ID":"ea32e654-51eb-42a0-a3ae-70119a46cec8","Type":"ContainerDied","Data":"6de3e1915d91626753c6ed1391484369da208523d20372efafbe555ce417caeb"}
Sep 16 14:28:12.374174 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.374141    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"]
Sep 16 14:28:12.379853 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.379831    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:12.382680 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.382655    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-d31d7c-dockercfg-prwp9\""
Sep 16 14:28:12.382840 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.382706    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-d31d7c\""
Sep 16 14:28:12.384238 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.384216    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"]
Sep 16 14:28:12.387886 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.387867    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:12.388961 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.388939    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"]
Sep 16 14:28:12.399442 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.399415    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"]
Sep 16 14:28:12.433785 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.433759    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"
Sep 16 14:28:12.511885 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.511857    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea32e654-51eb-42a0-a3ae-70119a46cec8-maas-api-service-ca\") pod \"ea32e654-51eb-42a0-a3ae-70119a46cec8\" (UID: \"ea32e654-51eb-42a0-a3ae-70119a46cec8\") "
Sep 16 14:28:12.512096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.512007    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf\" (UID: \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:12.512096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.512032    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txp9g\" (UniqueName: \"kubernetes.io/projected/e3d31b84-6f92-4825-b8c2-9fc7c314787c-kube-api-access-txp9g\") pod \"payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h\" (UID: \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:12.512096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.512063    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3d31b84-6f92-4825-b8c2-9fc7c314787c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h\" (UID: \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:12.512096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.512090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwmhm\" (UniqueName: \"kubernetes.io/projected/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-kube-api-access-cwmhm\") pod \"payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf\" (UID: \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:12.512320 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.512272    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea32e654-51eb-42a0-a3ae-70119a46cec8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ea32e654-51eb-42a0-a3ae-70119a46cec8" (UID: "ea32e654-51eb-42a0-a3ae-70119a46cec8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:28:12.612771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.612687    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3d31b84-6f92-4825-b8c2-9fc7c314787c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h\" (UID: \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:12.612771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.612738    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwmhm\" (UniqueName: \"kubernetes.io/projected/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-kube-api-access-cwmhm\") pod \"payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf\" (UID: \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:12.612942 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.612809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf\" (UID: \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:12.612942 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.612829    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txp9g\" (UniqueName: \"kubernetes.io/projected/e3d31b84-6f92-4825-b8c2-9fc7c314787c-kube-api-access-txp9g\") pod \"payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h\" (UID: \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:12.612942 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.612871    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea32e654-51eb-42a0-a3ae-70119a46cec8-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:12.613420 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.613403    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3d31b84-6f92-4825-b8c2-9fc7c314787c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h\" (UID: \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:12.613503 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.613486    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf\" (UID: \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:12.630522 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.630487    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txp9g\" (UniqueName: \"kubernetes.io/projected/e3d31b84-6f92-4825-b8c2-9fc7c314787c-kube-api-access-txp9g\") pod \"payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h\" (UID: \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:12.632519 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.632493    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwmhm\" (UniqueName: \"kubernetes.io/projected/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-kube-api-access-cwmhm\") pod \"payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf\" (UID: \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:12.692695 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.692666    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:12.727578 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.727534    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:12.964627 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:12.964604    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"]
Sep 16 14:28:12.966496 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:28:12.966473    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d31b84_6f92_4825_b8c2_9fc7c314787c.slice/crio-71a258d17d10aacf7b0245d220b41e105708c7e063e802e083faee8d2268b5a4 WatchSource:0}: Error finding container 71a258d17d10aacf7b0245d220b41e105708c7e063e802e083faee8d2268b5a4: Status 404 returned error can't find the container with id 71a258d17d10aacf7b0245d220b41e105708c7e063e802e083faee8d2268b5a4
Sep 16 14:28:13.009264 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.009241    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"]
Sep 16 14:28:13.010673 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:28:13.010651    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f40037_a3b8_49e2_b466_9f0c8a5ba99f.slice/crio-be25adeaf00a814f5228142c8a62c3b7d0acc972dc5b514cc42936f892b52d5f WatchSource:0}: Error finding container be25adeaf00a814f5228142c8a62c3b7d0acc972dc5b514cc42936f892b52d5f: Status 404 returned error can't find the container with id be25adeaf00a814f5228142c8a62c3b7d0acc972dc5b514cc42936f892b52d5f
Sep 16 14:28:13.296342 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.296257    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"
Sep 16 14:28:13.296342 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.296299    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf" event={"ID":"ea32e654-51eb-42a0-a3ae-70119a46cec8","Type":"ContainerDied","Data":"bff74c1238222d19f631ac6943db24df87f1a817614d6780b61dbe99312c40b9"}
Sep 16 14:28:13.296582 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.296350    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bff74c1238222d19f631ac6943db24df87f1a817614d6780b61dbe99312c40b9"
Sep 16 14:28:13.298166 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.298136    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h" event={"ID":"e3d31b84-6f92-4825-b8c2-9fc7c314787c","Type":"ContainerStarted","Data":"989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9"}
Sep 16 14:28:13.298316 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.298186    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h" event={"ID":"e3d31b84-6f92-4825-b8c2-9fc7c314787c","Type":"ContainerStarted","Data":"71a258d17d10aacf7b0245d220b41e105708c7e063e802e083faee8d2268b5a4"}
Sep 16 14:28:13.298316 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.298240    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:13.300015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.299988    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf" event={"ID":"76f40037-a3b8-49e2-b466-9f0c8a5ba99f","Type":"ContainerStarted","Data":"897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00"}
Sep 16 14:28:13.300126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.300037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf" event={"ID":"76f40037-a3b8-49e2-b466-9f0c8a5ba99f","Type":"ContainerStarted","Data":"be25adeaf00a814f5228142c8a62c3b7d0acc972dc5b514cc42936f892b52d5f"}
Sep 16 14:28:13.300126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.300088    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:13.318015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.317974    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h" podStartSLOduration=1.317959189 podStartE2EDuration="1.317959189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:13.316630199 +0000 UTC m=+1212.793862298" watchObservedRunningTime="2026-09-16 14:28:13.317959189 +0000 UTC m=+1212.795191263"
Sep 16 14:28:13.345143 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:13.345083    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf" podStartSLOduration=1.345063721 podStartE2EDuration="1.345063721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:13.343708128 +0000 UTC m=+1212.820940220" watchObservedRunningTime="2026-09-16 14:28:13.345063721 +0000 UTC m=+1212.822295797"
Sep 16 14:28:15.749356 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:15.749320    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"]
Sep 16 14:28:15.749841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:15.749517    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp" podUID="7bfdeeff-f76c-473c-8bd3-0558fb2739e3" containerName="payload-pre-processing" containerID="cri-o://72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4" gracePeriod=30
Sep 16 14:28:16.002739 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.002679    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"]
Sep 16 14:28:16.008785 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.008757    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:28:16.011576 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.011544    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5f53f6be-czjbf"]
Sep 16 14:28:16.147624 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.147592    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-plugins-config-volume\") pod \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\" (UID: \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\") "
Sep 16 14:28:16.147766 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.147647    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5btsk\" (UniqueName: \"kubernetes.io/projected/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-kube-api-access-5btsk\") pod \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\" (UID: \"7bfdeeff-f76c-473c-8bd3-0558fb2739e3\") "
Sep 16 14:28:16.147963 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.147936    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7bfdeeff-f76c-473c-8bd3-0558fb2739e3" (UID: "7bfdeeff-f76c-473c-8bd3-0558fb2739e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:28:16.149870 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.149836    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-kube-api-access-5btsk" (OuterVolumeSpecName: "kube-api-access-5btsk") pod "7bfdeeff-f76c-473c-8bd3-0558fb2739e3" (UID: "7bfdeeff-f76c-473c-8bd3-0558fb2739e3"). InnerVolumeSpecName "kube-api-access-5btsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:16.248500 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.248468    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:16.248500 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.248495    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5btsk\" (UniqueName: \"kubernetes.io/projected/7bfdeeff-f76c-473c-8bd3-0558fb2739e3-kube-api-access-5btsk\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:16.317439 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.317355    2566 generic.go:356] "Generic (PLEG): container finished" podID="7bfdeeff-f76c-473c-8bd3-0558fb2739e3" containerID="72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4" exitCode=0
Sep 16 14:28:16.317439 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.317396    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"
Sep 16 14:28:16.317624 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.317445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp" event={"ID":"7bfdeeff-f76c-473c-8bd3-0558fb2739e3","Type":"ContainerDied","Data":"72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4"}
Sep 16 14:28:16.317624 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.317490    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp" event={"ID":"7bfdeeff-f76c-473c-8bd3-0558fb2739e3","Type":"ContainerDied","Data":"c2a588a809f01283077816247cfdc5856adc0fc98a03c58c6d3dd2676a9bcfa8"}
Sep 16 14:28:16.317624 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.317512    2566 scope.go:117] "RemoveContainer" containerID="72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4"
Sep 16 14:28:16.326359 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.326342    2566 scope.go:117] "RemoveContainer" containerID="72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4"
Sep 16 14:28:16.326642 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:28:16.326620    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4\": container with ID starting with 72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4 not found: ID does not exist" containerID="72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4"
Sep 16 14:28:16.326742 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.326647    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4"} err="failed to get container status \"72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4\": rpc error: code = NotFound desc = could not find container \"72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4\": container with ID starting with 72fb60a6bf4ea33985e1e01219d9b9ce97d0bf6a4284d50fb130fb5fdfd201d4 not found: ID does not exist"
Sep 16 14:28:16.351184 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.351162    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"]
Sep 16 14:28:16.355385 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.355363    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f53f6be-7465fc47d6-sjvkp"]
Sep 16 14:28:16.984947 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.984907    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bfdeeff-f76c-473c-8bd3-0558fb2739e3" path="/var/lib/kubelet/pods/7bfdeeff-f76c-473c-8bd3-0558fb2739e3/volumes"
Sep 16 14:28:16.985425 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:16.985372    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea32e654-51eb-42a0-a3ae-70119a46cec8" path="/var/lib/kubelet/pods/ea32e654-51eb-42a0-a3ae-70119a46cec8/volumes"
Sep 16 14:28:24.307176 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.307139    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:28:24.307646 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.307602    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:28:24.984430 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.984402    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"]
Sep 16 14:28:24.984771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.984757    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bfdeeff-f76c-473c-8bd3-0558fb2739e3" containerName="payload-pre-processing"
Sep 16 14:28:24.984771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.984772    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bfdeeff-f76c-473c-8bd3-0558fb2739e3" containerName="payload-pre-processing"
Sep 16 14:28:24.984909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.984788    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea32e654-51eb-42a0-a3ae-70119a46cec8" containerName="revoke-keys"
Sep 16 14:28:24.984909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.984795    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea32e654-51eb-42a0-a3ae-70119a46cec8" containerName="revoke-keys"
Sep 16 14:28:24.984909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.984850    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea32e654-51eb-42a0-a3ae-70119a46cec8" containerName="revoke-keys"
Sep 16 14:28:24.984909 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.984858    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bfdeeff-f76c-473c-8bd3-0558fb2739e3" containerName="payload-pre-processing"
Sep 16 14:28:24.989262 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.989237    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"
Sep 16 14:28:24.991669 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.991648    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"]
Sep 16 14:28:24.991861 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.991843    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:28:24.991965 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:24.991844    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:28:25.132706 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:25.132667    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a41e58d-686f-4e22-bcea-67df32aae748-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm\" (UID: \"0a41e58d-686f-4e22-bcea-67df32aae748\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"
Sep 16 14:28:25.233536 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:25.233493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a41e58d-686f-4e22-bcea-67df32aae748-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm\" (UID: \"0a41e58d-686f-4e22-bcea-67df32aae748\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"
Sep 16 14:28:25.234146 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:25.234120    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a41e58d-686f-4e22-bcea-67df32aae748-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm\" (UID: \"0a41e58d-686f-4e22-bcea-67df32aae748\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"
Sep 16 14:28:25.301357 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:25.301265    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"
Sep 16 14:28:25.573238 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:25.573210    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"]
Sep 16 14:28:25.574811 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:28:25.574789    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a41e58d_686f_4e22_bcea_67df32aae748.slice/crio-a17a7bfc468db4320e7b7911917119b2f706b8c38f8673c1c836d3c84b91ab3a WatchSource:0}: Error finding container a17a7bfc468db4320e7b7911917119b2f706b8c38f8673c1c836d3c84b91ab3a: Status 404 returned error can't find the container with id a17a7bfc468db4320e7b7911917119b2f706b8c38f8673c1c836d3c84b91ab3a
Sep 16 14:28:26.372340 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:26.372300    2566 generic.go:356] "Generic (PLEG): container finished" podID="0a41e58d-686f-4e22-bcea-67df32aae748" containerID="b052330c5bdacc0429d44fd08285415960fe0fa91501ff79b05a0cbdde9c4277" exitCode=0
Sep 16 14:28:26.372491 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:26.372393    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm" event={"ID":"0a41e58d-686f-4e22-bcea-67df32aae748","Type":"ContainerDied","Data":"b052330c5bdacc0429d44fd08285415960fe0fa91501ff79b05a0cbdde9c4277"}
Sep 16 14:28:26.372491 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:26.372445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm" event={"ID":"0a41e58d-686f-4e22-bcea-67df32aae748","Type":"ContainerStarted","Data":"a17a7bfc468db4320e7b7911917119b2f706b8c38f8673c1c836d3c84b91ab3a"}
Sep 16 14:28:27.507030 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:27.507002    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"
Sep 16 14:28:27.657523 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:27.657449    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a41e58d-686f-4e22-bcea-67df32aae748-maas-api-service-ca\") pod \"0a41e58d-686f-4e22-bcea-67df32aae748\" (UID: \"0a41e58d-686f-4e22-bcea-67df32aae748\") "
Sep 16 14:28:27.657790 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:27.657766    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a41e58d-686f-4e22-bcea-67df32aae748-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a41e58d-686f-4e22-bcea-67df32aae748" (UID: "0a41e58d-686f-4e22-bcea-67df32aae748"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:28:27.758995 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:27.758957    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a41e58d-686f-4e22-bcea-67df32aae748-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:28.383590 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:28.383540    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"
Sep 16 14:28:28.383765 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:28.383610    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm" event={"ID":"0a41e58d-686f-4e22-bcea-67df32aae748","Type":"ContainerDied","Data":"a17a7bfc468db4320e7b7911917119b2f706b8c38f8673c1c836d3c84b91ab3a"}
Sep 16 14:28:28.383765 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:28.383657    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a17a7bfc468db4320e7b7911917119b2f706b8c38f8673c1c836d3c84b91ab3a"
Sep 16 14:28:29.866170 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:29.866136    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"]
Sep 16 14:28:29.875791 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:29.875760    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"]
Sep 16 14:28:33.785145 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.785114    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"]
Sep 16 14:28:33.785492 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.785484    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a41e58d-686f-4e22-bcea-67df32aae748" containerName="revoke-keys"
Sep 16 14:28:33.785537 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.785496    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a41e58d-686f-4e22-bcea-67df32aae748" containerName="revoke-keys"
Sep 16 14:28:33.785589 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.785556    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a41e58d-686f-4e22-bcea-67df32aae748" containerName="revoke-keys"
Sep 16 14:28:33.788562 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.788540    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.790955 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.790933    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-c0974e\"/\"auth-test-model-c0974e-kserve-self-signed-certs\""
Sep 16 14:28:33.790955 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.790949    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-c0974e\"/\"openshift-service-ca.crt\""
Sep 16 14:28:33.791724 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.791706    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-c0974e\"/\"default-dockercfg-cm94w\""
Sep 16 14:28:33.791849 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.791747    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-c0974e\"/\"kube-root-ca.crt\""
Sep 16 14:28:33.807893 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.807866    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"]
Sep 16 14:28:33.808315 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.808293    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-tmp-dir\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.808540 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.808521    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44b3506b-dda0-4fef-8a91-3504d1333b8d-tls-certs\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.808820 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.808796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-dshm\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.808920 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.808833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-home\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.808920 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.808856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-model-cache\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.808920 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.808893    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d7r5\" (UniqueName: \"kubernetes.io/projected/44b3506b-dda0-4fef-8a91-3504d1333b8d-kube-api-access-2d7r5\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.909440 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.909409    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-dshm\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.909440 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.909443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-home\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.909711 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.909457    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-model-cache\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.909711 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.909481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d7r5\" (UniqueName: \"kubernetes.io/projected/44b3506b-dda0-4fef-8a91-3504d1333b8d-kube-api-access-2d7r5\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.909711 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.909631    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-tmp-dir\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.909711 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.909696    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44b3506b-dda0-4fef-8a91-3504d1333b8d-tls-certs\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.909920 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.909899    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-home\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.910001 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.909984    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-model-cache\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.910049 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.910032    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-tmp-dir\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.912068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.912042    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-dshm\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.912221 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.912204    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44b3506b-dda0-4fef-8a91-3504d1333b8d-tls-certs\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:33.923938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:33.923911    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d7r5\" (UniqueName: \"kubernetes.io/projected/44b3506b-dda0-4fef-8a91-3504d1333b8d-kube-api-access-2d7r5\") pod \"auth-test-model-c0974e-kserve-65dd4f795f-9lvsv\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:34.098792 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:34.098700    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:34.370018 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:34.369994    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"]
Sep 16 14:28:34.372030 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:28:34.372009    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44b3506b_dda0_4fef_8a91_3504d1333b8d.slice/crio-f34fb82e2be3827df5d13cafb69180acef9ef5eb7fed00423863c0404fa53222 WatchSource:0}: Error finding container f34fb82e2be3827df5d13cafb69180acef9ef5eb7fed00423863c0404fa53222: Status 404 returned error can't find the container with id f34fb82e2be3827df5d13cafb69180acef9ef5eb7fed00423863c0404fa53222
Sep 16 14:28:34.414795 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:34.414768    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv" event={"ID":"44b3506b-dda0-4fef-8a91-3504d1333b8d","Type":"ContainerStarted","Data":"f34fb82e2be3827df5d13cafb69180acef9ef5eb7fed00423863c0404fa53222"}
Sep 16 14:28:34.892447 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:34.892410    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:28:35.423125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:35.423065    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv" event={"ID":"44b3506b-dda0-4fef-8a91-3504d1333b8d","Type":"ContainerStarted","Data":"d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8"}
Sep 16 14:28:35.451171 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:35.451098    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv" podStartSLOduration=2.451077182 podStartE2EDuration="2.451077182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:35.448863872 +0000 UTC m=+1234.926095949" watchObservedRunningTime="2026-09-16 14:28:35.451077182 +0000 UTC m=+1234.928309258"
Sep 16 14:28:41.255408 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:41.255378    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"]
Sep 16 14:28:41.264784 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:41.264759    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d48266-lgcqm"]
Sep 16 14:28:42.984697 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:42.984667    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a41e58d-686f-4e22-bcea-67df32aae748" path="/var/lib/kubelet/pods/0a41e58d-686f-4e22-bcea-67df32aae748/volumes"
Sep 16 14:28:44.099615 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:44.099541    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:44.100023 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:44.099641    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:44.112449 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:44.112426    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:44.482431 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:44.482390    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:28:44.886226 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:44.886110    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4" podUID="94b3006c-d63e-4618-b9ec-b6c54637b16f" containerName="main" containerID="cri-o://c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840" gracePeriod=45
Sep 16 14:28:44.896957 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:44.896922    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn" podUID="6b67adb9-90d6-4269-b109-084963bae30e" containerName="main" containerID="cri-o://4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a" gracePeriod=45
Sep 16 14:28:45.170861 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.170840    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:28:45.174273 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.174241    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:28:45.215311 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215283    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phqqq\" (UniqueName: \"kubernetes.io/projected/6b67adb9-90d6-4269-b109-084963bae30e-kube-api-access-phqqq\") pod \"6b67adb9-90d6-4269-b109-084963bae30e\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") "
Sep 16 14:28:45.215498 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215326    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-dshm\") pod \"94b3006c-d63e-4618-b9ec-b6c54637b16f\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") "
Sep 16 14:28:45.215498 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215412    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94b3006c-d63e-4618-b9ec-b6c54637b16f-tls-certs\") pod \"94b3006c-d63e-4618-b9ec-b6c54637b16f\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") "
Sep 16 14:28:45.215498 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215464    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngfgm\" (UniqueName: \"kubernetes.io/projected/94b3006c-d63e-4618-b9ec-b6c54637b16f-kube-api-access-ngfgm\") pod \"94b3006c-d63e-4618-b9ec-b6c54637b16f\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") "
Sep 16 14:28:45.215664 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215502    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-home\") pod \"6b67adb9-90d6-4269-b109-084963bae30e\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") "
Sep 16 14:28:45.215664 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215525    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-home\") pod \"94b3006c-d63e-4618-b9ec-b6c54637b16f\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") "
Sep 16 14:28:45.215664 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215597    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-model-cache\") pod \"94b3006c-d63e-4618-b9ec-b6c54637b16f\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") "
Sep 16 14:28:45.215664 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215651    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b67adb9-90d6-4269-b109-084963bae30e-tls-certs\") pod \"6b67adb9-90d6-4269-b109-084963bae30e\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") "
Sep 16 14:28:45.215860 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215689    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-dshm\") pod \"6b67adb9-90d6-4269-b109-084963bae30e\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") "
Sep 16 14:28:45.215860 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215730    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-tmp-dir\") pod \"94b3006c-d63e-4618-b9ec-b6c54637b16f\" (UID: \"94b3006c-d63e-4618-b9ec-b6c54637b16f\") "
Sep 16 14:28:45.215860 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215773    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-tmp-dir\") pod \"6b67adb9-90d6-4269-b109-084963bae30e\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") "
Sep 16 14:28:45.215860 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215794    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-model-cache\") pod \"6b67adb9-90d6-4269-b109-084963bae30e\" (UID: \"6b67adb9-90d6-4269-b109-084963bae30e\") "
Sep 16 14:28:45.216044 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.215873    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-home" (OuterVolumeSpecName: "home") pod "6b67adb9-90d6-4269-b109-084963bae30e" (UID: "6b67adb9-90d6-4269-b109-084963bae30e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.216650 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.216143    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.216650 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.216338    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-home" (OuterVolumeSpecName: "home") pod "94b3006c-d63e-4618-b9ec-b6c54637b16f" (UID: "94b3006c-d63e-4618-b9ec-b6c54637b16f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.216650 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.216491    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "94b3006c-d63e-4618-b9ec-b6c54637b16f" (UID: "94b3006c-d63e-4618-b9ec-b6c54637b16f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.216650 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.216506    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b67adb9-90d6-4269-b109-084963bae30e" (UID: "6b67adb9-90d6-4269-b109-084963bae30e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.216650 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.216594    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-model-cache" (OuterVolumeSpecName: "model-cache") pod "94b3006c-d63e-4618-b9ec-b6c54637b16f" (UID: "94b3006c-d63e-4618-b9ec-b6c54637b16f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.216650 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.216600    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b67adb9-90d6-4269-b109-084963bae30e" (UID: "6b67adb9-90d6-4269-b109-084963bae30e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.218812 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.218773    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b67adb9-90d6-4269-b109-084963bae30e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b67adb9-90d6-4269-b109-084963bae30e" (UID: "6b67adb9-90d6-4269-b109-084963bae30e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:28:45.218927 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.218830    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94b3006c-d63e-4618-b9ec-b6c54637b16f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "94b3006c-d63e-4618-b9ec-b6c54637b16f" (UID: "94b3006c-d63e-4618-b9ec-b6c54637b16f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:28:45.218927 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.218893    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-dshm" (OuterVolumeSpecName: "dshm") pod "94b3006c-d63e-4618-b9ec-b6c54637b16f" (UID: "94b3006c-d63e-4618-b9ec-b6c54637b16f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.219288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.219258    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94b3006c-d63e-4618-b9ec-b6c54637b16f-kube-api-access-ngfgm" (OuterVolumeSpecName: "kube-api-access-ngfgm") pod "94b3006c-d63e-4618-b9ec-b6c54637b16f" (UID: "94b3006c-d63e-4618-b9ec-b6c54637b16f"). InnerVolumeSpecName "kube-api-access-ngfgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:45.219487 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.219467    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-dshm" (OuterVolumeSpecName: "dshm") pod "6b67adb9-90d6-4269-b109-084963bae30e" (UID: "6b67adb9-90d6-4269-b109-084963bae30e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.219657 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.219638    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b67adb9-90d6-4269-b109-084963bae30e-kube-api-access-phqqq" (OuterVolumeSpecName: "kube-api-access-phqqq") pod "6b67adb9-90d6-4269-b109-084963bae30e" (UID: "6b67adb9-90d6-4269-b109-084963bae30e"). InnerVolumeSpecName "kube-api-access-phqqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:45.316822 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316779    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b67adb9-90d6-4269-b109-084963bae30e-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.316822 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316811    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.316822 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316823    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.316822 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316833    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.317100 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316844    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b67adb9-90d6-4269-b109-084963bae30e-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.317100 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316856    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phqqq\" (UniqueName: \"kubernetes.io/projected/6b67adb9-90d6-4269-b109-084963bae30e-kube-api-access-phqqq\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.317100 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316868    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.317100 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316878    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94b3006c-d63e-4618-b9ec-b6c54637b16f-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.317100 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316889    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngfgm\" (UniqueName: \"kubernetes.io/projected/94b3006c-d63e-4618-b9ec-b6c54637b16f-kube-api-access-ngfgm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.317100 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316898    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.317100 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.316910    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94b3006c-d63e-4618-b9ec-b6c54637b16f-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.472821 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.472791    2566 generic.go:356] "Generic (PLEG): container finished" podID="94b3006c-d63e-4618-b9ec-b6c54637b16f" containerID="c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840" exitCode=0
Sep 16 14:28:45.473000 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.472892    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"
Sep 16 14:28:45.473000 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.472932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4" event={"ID":"94b3006c-d63e-4618-b9ec-b6c54637b16f","Type":"ContainerDied","Data":"c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840"}
Sep 16 14:28:45.473000 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.472982    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4" event={"ID":"94b3006c-d63e-4618-b9ec-b6c54637b16f","Type":"ContainerDied","Data":"ea63ccecd39c95d73fdd20a7d412363ac1a4df14ccdee4c0c81550150538bd70"}
Sep 16 14:28:45.473251 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.473006    2566 scope.go:117] "RemoveContainer" containerID="c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840"
Sep 16 14:28:45.474475 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.474444    2566 generic.go:356] "Generic (PLEG): container finished" podID="6b67adb9-90d6-4269-b109-084963bae30e" containerID="4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a" exitCode=0
Sep 16 14:28:45.474608 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.474481    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"
Sep 16 14:28:45.474608 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.474516    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn" event={"ID":"6b67adb9-90d6-4269-b109-084963bae30e","Type":"ContainerDied","Data":"4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a"}
Sep 16 14:28:45.474608 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.474594    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn" event={"ID":"6b67adb9-90d6-4269-b109-084963bae30e","Type":"ContainerDied","Data":"d9004ac381316c8d3aeb0d91314ce6c4c3da30627a68059cd552c022b15fd2e5"}
Sep 16 14:28:45.483728 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.483713    2566 scope.go:117] "RemoveContainer" containerID="c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840"
Sep 16 14:28:45.484045 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:28:45.483978    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840\": container with ID starting with c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840 not found: ID does not exist" containerID="c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840"
Sep 16 14:28:45.484045 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.483998    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840"} err="failed to get container status \"c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840\": rpc error: code = NotFound desc = could not find container \"c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840\": container with ID starting with c6042c2031359e9638b21355ca00f4cf0cc3ab915cb73e3ffb804f22a15d9840 not found: ID does not exist"
Sep 16 14:28:45.484045 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.484013    2566 scope.go:117] "RemoveContainer" containerID="4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a"
Sep 16 14:28:45.492265 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.492249    2566 scope.go:117] "RemoveContainer" containerID="4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a"
Sep 16 14:28:45.492519 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:28:45.492499    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a\": container with ID starting with 4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a not found: ID does not exist" containerID="4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a"
Sep 16 14:28:45.492595 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.492527    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a"} err="failed to get container status \"4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a\": rpc error: code = NotFound desc = could not find container \"4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a\": container with ID starting with 4577fcccfbeaab26d7d3df412527a84b6cc277d26bdf33836a0af134c796f26a not found: ID does not exist"
Sep 16 14:28:45.505573 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.505550    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"]
Sep 16 14:28:45.508258 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.508238    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/premium-w3-d48266-kserve-645ff5669f-pwgc4"]
Sep 16 14:28:45.522684 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.522641    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"]
Sep 16 14:28:45.524298 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:45.524281    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/simulator-w3-d48266-kserve-768d8858c4-w8khn"]
Sep 16 14:28:46.984543 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:46.984505    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b67adb9-90d6-4269-b109-084963bae30e" path="/var/lib/kubelet/pods/6b67adb9-90d6-4269-b109-084963bae30e/volumes"
Sep 16 14:28:46.984923 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:28:46.984909    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94b3006c-d63e-4618-b9ec-b6c54637b16f" path="/var/lib/kubelet/pods/94b3006c-d63e-4618-b9ec-b6c54637b16f/volumes"
Sep 16 14:29:11.862087 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:29:11.862052    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:29:29.990960 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:29:29.990925    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:29:48.592073 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:29:48.592032    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:30:00.199499 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.199467    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt"]
Sep 16 14:30:00.199954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.199837    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b67adb9-90d6-4269-b109-084963bae30e" containerName="main"
Sep 16 14:30:00.199954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.199850    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b67adb9-90d6-4269-b109-084963bae30e" containerName="main"
Sep 16 14:30:00.199954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.199874    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94b3006c-d63e-4618-b9ec-b6c54637b16f" containerName="main"
Sep 16 14:30:00.199954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.199879    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94b3006c-d63e-4618-b9ec-b6c54637b16f" containerName="main"
Sep 16 14:30:00.199954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.199938    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="94b3006c-d63e-4618-b9ec-b6c54637b16f" containerName="main"
Sep 16 14:30:00.199954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.199946    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b67adb9-90d6-4269-b109-084963bae30e" containerName="main"
Sep 16 14:30:00.203234 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.203208    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q"]
Sep 16 14:30:00.203388 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.203332    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt"
Sep 16 14:30:00.206764 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.206747    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q"
Sep 16 14:30:00.207110 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.207095    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:30:00.214113 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.214096    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt"
Sep 16 14:30:00.220219 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.220200    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q"
Sep 16 14:30:00.224862 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.224844    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt"]
Sep 16 14:30:00.229036 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.229003    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q"]
Sep 16 14:30:00.495108 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.495081    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt"]
Sep 16 14:30:00.500733 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:30:00.500642    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ae966e_142a_4f19_ad27_af5dbbd8bbcb.slice/crio-c598b22ab6ef0b4264e6d3e0795cef5d1f2772c4af82425d4fbe2e25d359161b WatchSource:0}: Error finding container c598b22ab6ef0b4264e6d3e0795cef5d1f2772c4af82425d4fbe2e25d359161b: Status 404 returned error can't find the container with id c598b22ab6ef0b4264e6d3e0795cef5d1f2772c4af82425d4fbe2e25d359161b
Sep 16 14:30:00.503532 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.503503    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q"]
Sep 16 14:30:00.506907 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:30:00.506880    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e50b9c_a1b0_497e_8916_ffd32872e836.slice/crio-125feb529a3b45660bf13f9413d6e10273aa365ce2129804ba549de992d07c14 WatchSource:0}: Error finding container 125feb529a3b45660bf13f9413d6e10273aa365ce2129804ba549de992d07c14: Status 404 returned error can't find the container with id 125feb529a3b45660bf13f9413d6e10273aa365ce2129804ba549de992d07c14
Sep 16 14:30:00.855879 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.855833    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q" event={"ID":"53e50b9c-a1b0-497e-8916-ffd32872e836","Type":"ContainerStarted","Data":"e074ecae29b9a6ac4e8ee5b80321792488c91a0b36f94f204cd0df7d4d39fb95"}
Sep 16 14:30:00.856040 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.855901    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q" event={"ID":"53e50b9c-a1b0-497e-8916-ffd32872e836","Type":"ContainerStarted","Data":"125feb529a3b45660bf13f9413d6e10273aa365ce2129804ba549de992d07c14"}
Sep 16 14:30:00.857348 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.857323    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt" event={"ID":"17ae966e-142a-4f19-ad27-af5dbbd8bbcb","Type":"ContainerStarted","Data":"75df175f2f89d577bcdf61c98e22f86633aafc9cd6f19ab264b15e6d02c42d53"}
Sep 16 14:30:00.857438 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.857371    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt" event={"ID":"17ae966e-142a-4f19-ad27-af5dbbd8bbcb","Type":"ContainerStarted","Data":"c598b22ab6ef0b4264e6d3e0795cef5d1f2772c4af82425d4fbe2e25d359161b"}
Sep 16 14:30:00.873973 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.873926    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q" podStartSLOduration=0.873912764 podStartE2EDuration="873.912764ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:00.871983469 +0000 UTC m=+1320.349215567" watchObservedRunningTime="2026-09-16 14:30:00.873912764 +0000 UTC m=+1320.351144841"
Sep 16 14:30:00.887500 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:00.887455    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt" podStartSLOduration=0.887438062 podStartE2EDuration="887.438062ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:00.887317862 +0000 UTC m=+1320.364549937" watchObservedRunningTime="2026-09-16 14:30:00.887438062 +0000 UTC m=+1320.364670138"
Sep 16 14:30:01.863396 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:01.863357    2566 generic.go:356] "Generic (PLEG): container finished" podID="53e50b9c-a1b0-497e-8916-ffd32872e836" containerID="e074ecae29b9a6ac4e8ee5b80321792488c91a0b36f94f204cd0df7d4d39fb95" exitCode=0
Sep 16 14:30:01.863845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:01.863444    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q" event={"ID":"53e50b9c-a1b0-497e-8916-ffd32872e836","Type":"ContainerDied","Data":"e074ecae29b9a6ac4e8ee5b80321792488c91a0b36f94f204cd0df7d4d39fb95"}
Sep 16 14:30:01.864960 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:01.864939    2566 generic.go:356] "Generic (PLEG): container finished" podID="17ae966e-142a-4f19-ad27-af5dbbd8bbcb" containerID="75df175f2f89d577bcdf61c98e22f86633aafc9cd6f19ab264b15e6d02c42d53" exitCode=0
Sep 16 14:30:01.865043 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:01.865000    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt" event={"ID":"17ae966e-142a-4f19-ad27-af5dbbd8bbcb","Type":"ContainerDied","Data":"75df175f2f89d577bcdf61c98e22f86633aafc9cd6f19ab264b15e6d02c42d53"}
Sep 16 14:30:03.024158 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:03.024132    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q"
Sep 16 14:30:03.029851 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:03.029832    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt"
Sep 16 14:30:03.877472 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:03.877437    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q"
Sep 16 14:30:03.877472 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:03.877453    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q" event={"ID":"53e50b9c-a1b0-497e-8916-ffd32872e836","Type":"ContainerDied","Data":"125feb529a3b45660bf13f9413d6e10273aa365ce2129804ba549de992d07c14"}
Sep 16 14:30:03.877779 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:03.877502    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="125feb529a3b45660bf13f9413d6e10273aa365ce2129804ba549de992d07c14"
Sep 16 14:30:03.878842 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:03.878820    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt" event={"ID":"17ae966e-142a-4f19-ad27-af5dbbd8bbcb","Type":"ContainerDied","Data":"c598b22ab6ef0b4264e6d3e0795cef5d1f2772c4af82425d4fbe2e25d359161b"}
Sep 16 14:30:03.878842 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:03.878839    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826150-wptjt"
Sep 16 14:30:03.878978 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:03.878861    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c598b22ab6ef0b4264e6d3e0795cef5d1f2772c4af82425d4fbe2e25d359161b"
Sep 16 14:30:11.405196 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:11.405162    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:30:30.994838 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:30.994762    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:30:55.434112 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:30:55.434076    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:31:08.763410 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:08.763357    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"]
Sep 16 14:31:08.781070 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:08.781031    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"]
Sep 16 14:31:08.825892 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:08.825855    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"]
Sep 16 14:31:18.240892 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:18.240852    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q"]
Sep 16 14:31:18.245841 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:18.245816    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-11636c-29826150-q7m9q"]
Sep 16 14:31:18.984526 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:18.984490    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53e50b9c-a1b0-497e-8916-ffd32872e836" path="/var/lib/kubelet/pods/53e50b9c-a1b0-497e-8916-ffd32872e836/volumes"
Sep 16 14:31:21.992862 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:21.992825    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:31:23.783244 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:23.783206    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn" podUID="19149d56-37f9-48af-842a-a7c3e00e8fa8" containerName="main" containerID="cri-o://8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b" gracePeriod=45
Sep 16 14:31:23.801000 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:23.800970    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2" podUID="a7d367e6-f559-4c22-b974-f75e645b5fbc" containerName="main" containerID="cri-o://d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc" gracePeriod=45
Sep 16 14:31:23.844456 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:23.844411    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs" podUID="004dda30-a002-46e8-8322-c5f543826458" containerName="main" containerID="cri-o://4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295" gracePeriod=45
Sep 16 14:31:24.105429 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.105394    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:31:24.126201 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.126172    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:31:24.131904 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.131883    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:31:24.251584 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251540    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-home\") pod \"a7d367e6-f559-4c22-b974-f75e645b5fbc\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") "
Sep 16 14:31:24.251735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251588    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-dshm\") pod \"19149d56-37f9-48af-842a-a7c3e00e8fa8\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") "
Sep 16 14:31:24.251735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251616    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d367e6-f559-4c22-b974-f75e645b5fbc-tls-certs\") pod \"a7d367e6-f559-4c22-b974-f75e645b5fbc\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") "
Sep 16 14:31:24.251735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251652    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-tmp-dir\") pod \"a7d367e6-f559-4c22-b974-f75e645b5fbc\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") "
Sep 16 14:31:24.251735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251680    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-model-cache\") pod \"a7d367e6-f559-4c22-b974-f75e645b5fbc\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") "
Sep 16 14:31:24.251735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251707    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/004dda30-a002-46e8-8322-c5f543826458-tls-certs\") pod \"004dda30-a002-46e8-8322-c5f543826458\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") "
Sep 16 14:31:24.251735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251730    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m94r\" (UniqueName: \"kubernetes.io/projected/19149d56-37f9-48af-842a-a7c3e00e8fa8-kube-api-access-5m94r\") pod \"19149d56-37f9-48af-842a-a7c3e00e8fa8\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") "
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251767    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-home\") pod \"004dda30-a002-46e8-8322-c5f543826458\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") "
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251807    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-model-cache\") pod \"004dda30-a002-46e8-8322-c5f543826458\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") "
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251827    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-dshm\") pod \"004dda30-a002-46e8-8322-c5f543826458\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") "
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251866    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19149d56-37f9-48af-842a-a7c3e00e8fa8-tls-certs\") pod \"19149d56-37f9-48af-842a-a7c3e00e8fa8\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") "
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251865    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-home" (OuterVolumeSpecName: "home") pod "a7d367e6-f559-4c22-b974-f75e645b5fbc" (UID: "a7d367e6-f559-4c22-b974-f75e645b5fbc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251896    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj4qx\" (UniqueName: \"kubernetes.io/projected/a7d367e6-f559-4c22-b974-f75e645b5fbc-kube-api-access-kj4qx\") pod \"a7d367e6-f559-4c22-b974-f75e645b5fbc\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") "
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251930    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-home\") pod \"19149d56-37f9-48af-842a-a7c3e00e8fa8\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") "
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251935    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a7d367e6-f559-4c22-b974-f75e645b5fbc" (UID: "a7d367e6-f559-4c22-b974-f75e645b5fbc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.251963    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-dshm\") pod \"a7d367e6-f559-4c22-b974-f75e645b5fbc\" (UID: \"a7d367e6-f559-4c22-b974-f75e645b5fbc\") "
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252021    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-model-cache\") pod \"19149d56-37f9-48af-842a-a7c3e00e8fa8\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") "
Sep 16 14:31:24.252068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252065    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-tmp-dir\") pod \"19149d56-37f9-48af-842a-a7c3e00e8fa8\" (UID: \"19149d56-37f9-48af-842a-a7c3e00e8fa8\") "
Sep 16 14:31:24.252652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252092    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-996jj\" (UniqueName: \"kubernetes.io/projected/004dda30-a002-46e8-8322-c5f543826458-kube-api-access-996jj\") pod \"004dda30-a002-46e8-8322-c5f543826458\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") "
Sep 16 14:31:24.252652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252138    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-tmp-dir\") pod \"004dda30-a002-46e8-8322-c5f543826458\" (UID: \"004dda30-a002-46e8-8322-c5f543826458\") "
Sep 16 14:31:24.252652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252492    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.252652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252512    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.253126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252067    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-home" (OuterVolumeSpecName: "home") pod "004dda30-a002-46e8-8322-c5f543826458" (UID: "004dda30-a002-46e8-8322-c5f543826458"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.253126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252234    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-model-cache" (OuterVolumeSpecName: "model-cache") pod "a7d367e6-f559-4c22-b974-f75e645b5fbc" (UID: "a7d367e6-f559-4c22-b974-f75e645b5fbc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.253126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252438    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-model-cache" (OuterVolumeSpecName: "model-cache") pod "004dda30-a002-46e8-8322-c5f543826458" (UID: "004dda30-a002-46e8-8322-c5f543826458"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.253126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252884    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "004dda30-a002-46e8-8322-c5f543826458" (UID: "004dda30-a002-46e8-8322-c5f543826458"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.253126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.252998    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19149d56-37f9-48af-842a-a7c3e00e8fa8" (UID: "19149d56-37f9-48af-842a-a7c3e00e8fa8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.253126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.253079    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-model-cache" (OuterVolumeSpecName: "model-cache") pod "19149d56-37f9-48af-842a-a7c3e00e8fa8" (UID: "19149d56-37f9-48af-842a-a7c3e00e8fa8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.253885 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.253370    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-home" (OuterVolumeSpecName: "home") pod "19149d56-37f9-48af-842a-a7c3e00e8fa8" (UID: "19149d56-37f9-48af-842a-a7c3e00e8fa8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.254795 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.254755    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7d367e6-f559-4c22-b974-f75e645b5fbc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a7d367e6-f559-4c22-b974-f75e645b5fbc" (UID: "a7d367e6-f559-4c22-b974-f75e645b5fbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:31:24.254795 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.254768    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/004dda30-a002-46e8-8322-c5f543826458-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "004dda30-a002-46e8-8322-c5f543826458" (UID: "004dda30-a002-46e8-8322-c5f543826458"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:31:24.255153 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.255130    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-dshm" (OuterVolumeSpecName: "dshm") pod "19149d56-37f9-48af-842a-a7c3e00e8fa8" (UID: "19149d56-37f9-48af-842a-a7c3e00e8fa8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.255240 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.255222    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19149d56-37f9-48af-842a-a7c3e00e8fa8-kube-api-access-5m94r" (OuterVolumeSpecName: "kube-api-access-5m94r") pod "19149d56-37f9-48af-842a-a7c3e00e8fa8" (UID: "19149d56-37f9-48af-842a-a7c3e00e8fa8"). InnerVolumeSpecName "kube-api-access-5m94r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:24.255300 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.255228    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-dshm" (OuterVolumeSpecName: "dshm") pod "a7d367e6-f559-4c22-b974-f75e645b5fbc" (UID: "a7d367e6-f559-4c22-b974-f75e645b5fbc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.255469 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.255446    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/004dda30-a002-46e8-8322-c5f543826458-kube-api-access-996jj" (OuterVolumeSpecName: "kube-api-access-996jj") pod "004dda30-a002-46e8-8322-c5f543826458" (UID: "004dda30-a002-46e8-8322-c5f543826458"). InnerVolumeSpecName "kube-api-access-996jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:24.255940 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.255916    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-dshm" (OuterVolumeSpecName: "dshm") pod "004dda30-a002-46e8-8322-c5f543826458" (UID: "004dda30-a002-46e8-8322-c5f543826458"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.256152 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.256128    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7d367e6-f559-4c22-b974-f75e645b5fbc-kube-api-access-kj4qx" (OuterVolumeSpecName: "kube-api-access-kj4qx") pod "a7d367e6-f559-4c22-b974-f75e645b5fbc" (UID: "a7d367e6-f559-4c22-b974-f75e645b5fbc"). InnerVolumeSpecName "kube-api-access-kj4qx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:24.256603 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.256547    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19149d56-37f9-48af-842a-a7c3e00e8fa8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19149d56-37f9-48af-842a-a7c3e00e8fa8" (UID: "19149d56-37f9-48af-842a-a7c3e00e8fa8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:31:24.281944 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.281921    2566 generic.go:356] "Generic (PLEG): container finished" podID="19149d56-37f9-48af-842a-a7c3e00e8fa8" containerID="8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b" exitCode=0
Sep 16 14:31:24.282062 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.281961    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"
Sep 16 14:31:24.282062 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.282006    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn" event={"ID":"19149d56-37f9-48af-842a-a7c3e00e8fa8","Type":"ContainerDied","Data":"8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b"}
Sep 16 14:31:24.282062 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.282058    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn" event={"ID":"19149d56-37f9-48af-842a-a7c3e00e8fa8","Type":"ContainerDied","Data":"54934de1afee0963876e8104715437e658ccc461bf4755f626be027afd323eba"}
Sep 16 14:31:24.282173 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.282085    2566 scope.go:117] "RemoveContainer" containerID="8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b"
Sep 16 14:31:24.283376 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.283352    2566 generic.go:356] "Generic (PLEG): container finished" podID="a7d367e6-f559-4c22-b974-f75e645b5fbc" containerID="d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc" exitCode=0
Sep 16 14:31:24.283456 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.283392    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"
Sep 16 14:31:24.283456 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.283436    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2" event={"ID":"a7d367e6-f559-4c22-b974-f75e645b5fbc","Type":"ContainerDied","Data":"d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc"}
Sep 16 14:31:24.283543 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.283468    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2" event={"ID":"a7d367e6-f559-4c22-b974-f75e645b5fbc","Type":"ContainerDied","Data":"a89cb5f5aea250192ff4e96905a3e6f9d4234388d1c205f0db2115b21f476e4b"}
Sep 16 14:31:24.284965 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.284940    2566 generic.go:356] "Generic (PLEG): container finished" podID="004dda30-a002-46e8-8322-c5f543826458" containerID="4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295" exitCode=0
Sep 16 14:31:24.285084 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.284975    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"
Sep 16 14:31:24.285084 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.285018    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs" event={"ID":"004dda30-a002-46e8-8322-c5f543826458","Type":"ContainerDied","Data":"4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295"}
Sep 16 14:31:24.285084 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.285049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs" event={"ID":"004dda30-a002-46e8-8322-c5f543826458","Type":"ContainerDied","Data":"a84ddcb58b0ea8db10dd4f0a9306c60dd81a0b84c853d384750b387990a204f6"}
Sep 16 14:31:24.299288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.298007    2566 scope.go:117] "RemoveContainer" containerID="8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b"
Sep 16 14:31:24.299288 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:31:24.298448    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b\": container with ID starting with 8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b not found: ID does not exist" containerID="8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b"
Sep 16 14:31:24.299288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.298486    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b"} err="failed to get container status \"8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b\": rpc error: code = NotFound desc = could not find container \"8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b\": container with ID starting with 8a2bc2c04e1426e1cfce8d8c6d49524a8177f398420ea4aa1659ffcd78742b4b not found: ID does not exist"
Sep 16 14:31:24.299288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.298509    2566 scope.go:117] "RemoveContainer" containerID="d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc"
Sep 16 14:31:24.308434 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.308416    2566 scope.go:117] "RemoveContainer" containerID="d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc"
Sep 16 14:31:24.308757 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:31:24.308738    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc\": container with ID starting with d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc not found: ID does not exist" containerID="d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc"
Sep 16 14:31:24.308838 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.308767    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc"} err="failed to get container status \"d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc\": rpc error: code = NotFound desc = could not find container \"d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc\": container with ID starting with d235583f082cfd3e39be4ca437c254f2236a9c607e835d8fd4caa502b22ed0dc not found: ID does not exist"
Sep 16 14:31:24.308838 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.308787    2566 scope.go:117] "RemoveContainer" containerID="4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295"
Sep 16 14:31:24.316425 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.316406    2566 scope.go:117] "RemoveContainer" containerID="4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295"
Sep 16 14:31:24.316698 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:31:24.316677    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295\": container with ID starting with 4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295 not found: ID does not exist" containerID="4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295"
Sep 16 14:31:24.316760 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.316705    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295"} err="failed to get container status \"4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295\": rpc error: code = NotFound desc = could not find container \"4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295\": container with ID starting with 4d7192e5d650b2342022c1795ce7da8ff1045255dfa7b7c5ed1bbb51fd394295 not found: ID does not exist"
Sep 16 14:31:24.323971 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.323948    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"]
Sep 16 14:31:24.330886 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.330866    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/simulator-w0-11636c-kserve-7ddd965c59-q5mrs"]
Sep 16 14:31:24.348859 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.348824    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"]
Sep 16 14:31:24.352933 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.352914    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19149d56-37f9-48af-842a-a7c3e00e8fa8-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.352936    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj4qx\" (UniqueName: \"kubernetes.io/projected/a7d367e6-f559-4c22-b974-f75e645b5fbc-kube-api-access-kj4qx\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.352946    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.352956    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.352963    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.352971    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.352986    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-996jj\" (UniqueName: \"kubernetes.io/projected/004dda30-a002-46e8-8322-c5f543826458-kube-api-access-996jj\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.352995    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.353003    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19149d56-37f9-48af-842a-a7c3e00e8fa8-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.353011    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d367e6-f559-4c22-b974-f75e645b5fbc-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353015 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.353018    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7d367e6-f559-4c22-b974-f75e645b5fbc-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.353025    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/004dda30-a002-46e8-8322-c5f543826458-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.353032    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m94r\" (UniqueName: \"kubernetes.io/projected/19149d56-37f9-48af-842a-a7c3e00e8fa8-kube-api-access-5m94r\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.353039    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.353047    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.353317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.353054    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/004dda30-a002-46e8-8322-c5f543826458-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.354141 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.354123    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-w0-11636c-kserve-5db449746f-wmvt2"]
Sep 16 14:31:24.369601 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.369579    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"]
Sep 16 14:31:24.373454 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.373433    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/distinct-2-w0-11636c-kserve-fb744bb6b-5dnjn"]
Sep 16 14:31:24.983686 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.983654    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="004dda30-a002-46e8-8322-c5f543826458" path="/var/lib/kubelet/pods/004dda30-a002-46e8-8322-c5f543826458/volumes"
Sep 16 14:31:24.984076 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.984024    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19149d56-37f9-48af-842a-a7c3e00e8fa8" path="/var/lib/kubelet/pods/19149d56-37f9-48af-842a-a7c3e00e8fa8/volumes"
Sep 16 14:31:24.984357 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:24.984345    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7d367e6-f559-4c22-b974-f75e645b5fbc" path="/var/lib/kubelet/pods/a7d367e6-f559-4c22-b974-f75e645b5fbc/volumes"
Sep 16 14:31:42.109456 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:42.109422    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:31:58.090694 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:31:58.090616    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:32:13.522628 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:32:13.522591    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:32:29.713814 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:32:29.713777    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:32:43.761589 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:32:43.761543    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:32:55.422880 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:32:55.422843    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:33:01.093467 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:01.093432    2566 scope.go:117] "RemoveContainer" containerID="adb86038750e963e7f459068e6019eda024957ce1a5d6a50d20d89ade3c70733"
Sep 16 14:33:01.101787 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:01.101766    2566 scope.go:117] "RemoveContainer" containerID="bc80e0e670d2d659ff75dcdd9c611d61083483c949baf1579f056a7195a2e11a"
Sep 16 14:33:01.962991 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:01.962955    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"]
Sep 16 14:33:14.108853 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:14.108817    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:33:15.613945 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.613914    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"]
Sep 16 14:33:15.614294 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614244    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7d367e6-f559-4c22-b974-f75e645b5fbc" containerName="main"
Sep 16 14:33:15.614294 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614255    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7d367e6-f559-4c22-b974-f75e645b5fbc" containerName="main"
Sep 16 14:33:15.614294 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614266    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53e50b9c-a1b0-497e-8916-ffd32872e836" containerName="cleanup"
Sep 16 14:33:15.614294 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614271    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="53e50b9c-a1b0-497e-8916-ffd32872e836" containerName="cleanup"
Sep 16 14:33:15.614294 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614279    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="004dda30-a002-46e8-8322-c5f543826458" containerName="main"
Sep 16 14:33:15.614294 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614285    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="004dda30-a002-46e8-8322-c5f543826458" containerName="main"
Sep 16 14:33:15.614483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614299    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19149d56-37f9-48af-842a-a7c3e00e8fa8" containerName="main"
Sep 16 14:33:15.614483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614304    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="19149d56-37f9-48af-842a-a7c3e00e8fa8" containerName="main"
Sep 16 14:33:15.614483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614315    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17ae966e-142a-4f19-ad27-af5dbbd8bbcb" containerName="cleanup"
Sep 16 14:33:15.614483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614320    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="17ae966e-142a-4f19-ad27-af5dbbd8bbcb" containerName="cleanup"
Sep 16 14:33:15.614483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614378    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="004dda30-a002-46e8-8322-c5f543826458" containerName="main"
Sep 16 14:33:15.614483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614388    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="53e50b9c-a1b0-497e-8916-ffd32872e836" containerName="cleanup"
Sep 16 14:33:15.614483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614396    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="17ae966e-142a-4f19-ad27-af5dbbd8bbcb" containerName="cleanup"
Sep 16 14:33:15.614483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614401    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7d367e6-f559-4c22-b974-f75e645b5fbc" containerName="main"
Sep 16 14:33:15.614483 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.614407    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="19149d56-37f9-48af-842a-a7c3e00e8fa8" containerName="main"
Sep 16 14:33:15.618772 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.618751    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.622439 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.622421    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-c0974e\"/\"sub-test-model-c0974e-kserve-self-signed-certs\""
Sep 16 14:33:15.628532 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.628509    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"]
Sep 16 14:33:15.710934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.710903    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/007bd32d-4705-4b03-a755-7aa08ff92e0b-tls-certs\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.710934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.710937    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-home\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.711140 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.710965    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-dshm\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.711140 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.711055    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-tmp-dir\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.711140 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.711092    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-model-cache\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.711252 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.711153    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swfnb\" (UniqueName: \"kubernetes.io/projected/007bd32d-4705-4b03-a755-7aa08ff92e0b-kube-api-access-swfnb\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.812341 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.812299    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-dshm\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.812512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.812360    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-tmp-dir\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.812512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.812381    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-model-cache\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.812512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.812413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swfnb\" (UniqueName: \"kubernetes.io/projected/007bd32d-4705-4b03-a755-7aa08ff92e0b-kube-api-access-swfnb\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.812512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.812484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/007bd32d-4705-4b03-a755-7aa08ff92e0b-tls-certs\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.812771 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.812526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-home\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.812840 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.812822    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-model-cache\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.812901 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.812873    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-home\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.812901 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.812888    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-tmp-dir\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.814838 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.814815    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-dshm\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.814978 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.814959    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/007bd32d-4705-4b03-a755-7aa08ff92e0b-tls-certs\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.820851 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.820834    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swfnb\" (UniqueName: \"kubernetes.io/projected/007bd32d-4705-4b03-a755-7aa08ff92e0b-kube-api-access-swfnb\") pod \"sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:15.927863 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:15.927788    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:16.215862 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:16.215787    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"]
Sep 16 14:33:16.218333 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:33:16.218298    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod007bd32d_4705_4b03_a755_7aa08ff92e0b.slice/crio-2f4287234b5d6a9887df72b46f7f9d3b4b3b8d4acb007c6ec3093ca22fefc29a WatchSource:0}: Error finding container 2f4287234b5d6a9887df72b46f7f9d3b4b3b8d4acb007c6ec3093ca22fefc29a: Status 404 returned error can't find the container with id 2f4287234b5d6a9887df72b46f7f9d3b4b3b8d4acb007c6ec3093ca22fefc29a
Sep 16 14:33:16.845420 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:16.845386    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd" event={"ID":"007bd32d-4705-4b03-a755-7aa08ff92e0b","Type":"ContainerStarted","Data":"9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c"}
Sep 16 14:33:16.845889 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:16.845436    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd" event={"ID":"007bd32d-4705-4b03-a755-7aa08ff92e0b","Type":"ContainerStarted","Data":"2f4287234b5d6a9887df72b46f7f9d3b4b3b8d4acb007c6ec3093ca22fefc29a"}
Sep 16 14:33:16.871024 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:16.870965    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd" podStartSLOduration=1.8709487729999998 podStartE2EDuration="1.870948773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:16.870505415 +0000 UTC m=+1516.347737506" watchObservedRunningTime="2026-09-16 14:33:16.870948773 +0000 UTC m=+1516.348180877"
Sep 16 14:33:16.982058 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:16.982017    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv" podUID="44b3506b-dda0-4fef-8a91-3504d1333b8d" containerName="main" containerID="cri-o://d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8" gracePeriod=45
Sep 16 14:33:17.219090 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.219067    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:33:17.324700 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.324664    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-model-cache\") pod \"44b3506b-dda0-4fef-8a91-3504d1333b8d\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") "
Sep 16 14:33:17.324700 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.324704    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-home\") pod \"44b3506b-dda0-4fef-8a91-3504d1333b8d\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") "
Sep 16 14:33:17.324956 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.324732    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2d7r5\" (UniqueName: \"kubernetes.io/projected/44b3506b-dda0-4fef-8a91-3504d1333b8d-kube-api-access-2d7r5\") pod \"44b3506b-dda0-4fef-8a91-3504d1333b8d\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") "
Sep 16 14:33:17.324956 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.324774    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-tmp-dir\") pod \"44b3506b-dda0-4fef-8a91-3504d1333b8d\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") "
Sep 16 14:33:17.324956 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.324812    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-dshm\") pod \"44b3506b-dda0-4fef-8a91-3504d1333b8d\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") "
Sep 16 14:33:17.324956 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.324838    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44b3506b-dda0-4fef-8a91-3504d1333b8d-tls-certs\") pod \"44b3506b-dda0-4fef-8a91-3504d1333b8d\" (UID: \"44b3506b-dda0-4fef-8a91-3504d1333b8d\") "
Sep 16 14:33:17.325140 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.324955    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-model-cache" (OuterVolumeSpecName: "model-cache") pod "44b3506b-dda0-4fef-8a91-3504d1333b8d" (UID: "44b3506b-dda0-4fef-8a91-3504d1333b8d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:33:17.325140 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.324973    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-home" (OuterVolumeSpecName: "home") pod "44b3506b-dda0-4fef-8a91-3504d1333b8d" (UID: "44b3506b-dda0-4fef-8a91-3504d1333b8d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:33:17.325140 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.325041    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44b3506b-dda0-4fef-8a91-3504d1333b8d" (UID: "44b3506b-dda0-4fef-8a91-3504d1333b8d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:33:17.325240 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.325180    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:33:17.325240 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.325198    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:33:17.325240 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.325211    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:33:17.327137 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.327115    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-dshm" (OuterVolumeSpecName: "dshm") pod "44b3506b-dda0-4fef-8a91-3504d1333b8d" (UID: "44b3506b-dda0-4fef-8a91-3504d1333b8d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:33:17.327250 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.327167    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44b3506b-dda0-4fef-8a91-3504d1333b8d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44b3506b-dda0-4fef-8a91-3504d1333b8d" (UID: "44b3506b-dda0-4fef-8a91-3504d1333b8d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:33:17.327250 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.327215    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44b3506b-dda0-4fef-8a91-3504d1333b8d-kube-api-access-2d7r5" (OuterVolumeSpecName: "kube-api-access-2d7r5") pod "44b3506b-dda0-4fef-8a91-3504d1333b8d" (UID: "44b3506b-dda0-4fef-8a91-3504d1333b8d"). InnerVolumeSpecName "kube-api-access-2d7r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:33:17.426615 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.426513    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d7r5\" (UniqueName: \"kubernetes.io/projected/44b3506b-dda0-4fef-8a91-3504d1333b8d-kube-api-access-2d7r5\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:33:17.426615 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.426540    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b3506b-dda0-4fef-8a91-3504d1333b8d-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:33:17.426615 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.426551    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44b3506b-dda0-4fef-8a91-3504d1333b8d-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:33:17.851663 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.851627    2566 generic.go:356] "Generic (PLEG): container finished" podID="44b3506b-dda0-4fef-8a91-3504d1333b8d" containerID="d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8" exitCode=0
Sep 16 14:33:17.852129 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.851674    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"
Sep 16 14:33:17.852129 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.851720    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv" event={"ID":"44b3506b-dda0-4fef-8a91-3504d1333b8d","Type":"ContainerDied","Data":"d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8"}
Sep 16 14:33:17.852129 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.851774    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv" event={"ID":"44b3506b-dda0-4fef-8a91-3504d1333b8d","Type":"ContainerDied","Data":"f34fb82e2be3827df5d13cafb69180acef9ef5eb7fed00423863c0404fa53222"}
Sep 16 14:33:17.852129 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.851801    2566 scope.go:117] "RemoveContainer" containerID="d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8"
Sep 16 14:33:17.861467 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.861451    2566 scope.go:117] "RemoveContainer" containerID="d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8"
Sep 16 14:33:17.861764 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:33:17.861748    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8\": container with ID starting with d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8 not found: ID does not exist" containerID="d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8"
Sep 16 14:33:17.861836 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.861776    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8"} err="failed to get container status \"d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8\": rpc error: code = NotFound desc = could not find container \"d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8\": container with ID starting with d7032e2ab27bf14006f0f4a4354f1ebeab241757a719d0cf1078dad2e39cdbd8 not found: ID does not exist"
Sep 16 14:33:17.874868 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.874849    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"]
Sep 16 14:33:17.880262 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:17.880242    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/auth-test-model-c0974e-kserve-65dd4f795f-9lvsv"]
Sep 16 14:33:18.983951 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:18.983916    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44b3506b-dda0-4fef-8a91-3504d1333b8d" path="/var/lib/kubelet/pods/44b3506b-dda0-4fef-8a91-3504d1333b8d/volumes"
Sep 16 14:33:25.928793 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:25.928707    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:25.929218 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:25.928964    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:25.941502 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:25.941480    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:26.910473 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:26.910441    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:33:29.113534 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:29.113501    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:33:33.091506 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.091469    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"]
Sep 16 14:33:33.091990 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.091849    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b3506b-dda0-4fef-8a91-3504d1333b8d" containerName="main"
Sep 16 14:33:33.091990 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.091865    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b3506b-dda0-4fef-8a91-3504d1333b8d" containerName="main"
Sep 16 14:33:33.091990 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.091954    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="44b3506b-dda0-4fef-8a91-3504d1333b8d" containerName="main"
Sep 16 14:33:33.094929 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.094910    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:33.097394 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.097369    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e9829ef2-dockercfg-tp49m\""
Sep 16 14:33:33.097580 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.097545    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e9829ef2\""
Sep 16 14:33:33.105852 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.105832    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"]
Sep 16 14:33:33.130195 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.130165    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"]
Sep 16 14:33:33.133612 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.133593    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:33.146719 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.146697    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"]
Sep 16 14:33:33.159372 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.159349    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b52332-c299-4f36-9e39-7f78981d9dce-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr\" (UID: \"28b52332-c299-4f36-9e39-7f78981d9dce\") " pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:33.159497 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.159437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb31742-33d3-4d73-88e2-644e20a6ef4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd\" (UID: \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:33.159497 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.159470    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv2lk\" (UniqueName: \"kubernetes.io/projected/9fb31742-33d3-4d73-88e2-644e20a6ef4a-kube-api-access-hv2lk\") pod \"payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd\" (UID: \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:33.159625 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.159494    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk5c2\" (UniqueName: \"kubernetes.io/projected/28b52332-c299-4f36-9e39-7f78981d9dce-kube-api-access-tk5c2\") pod \"payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr\" (UID: \"28b52332-c299-4f36-9e39-7f78981d9dce\") " pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:33.260143 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.260109    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb31742-33d3-4d73-88e2-644e20a6ef4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd\" (UID: \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:33.260143 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.260147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv2lk\" (UniqueName: \"kubernetes.io/projected/9fb31742-33d3-4d73-88e2-644e20a6ef4a-kube-api-access-hv2lk\") pod \"payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd\" (UID: \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:33.260371 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.260162    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk5c2\" (UniqueName: \"kubernetes.io/projected/28b52332-c299-4f36-9e39-7f78981d9dce-kube-api-access-tk5c2\") pod \"payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr\" (UID: \"28b52332-c299-4f36-9e39-7f78981d9dce\") " pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:33.260371 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.260215    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b52332-c299-4f36-9e39-7f78981d9dce-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr\" (UID: \"28b52332-c299-4f36-9e39-7f78981d9dce\") " pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:33.260889 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.260869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb31742-33d3-4d73-88e2-644e20a6ef4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd\" (UID: \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:33.260889 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.260879    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b52332-c299-4f36-9e39-7f78981d9dce-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr\" (UID: \"28b52332-c299-4f36-9e39-7f78981d9dce\") " pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:33.269613 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.269587    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk5c2\" (UniqueName: \"kubernetes.io/projected/28b52332-c299-4f36-9e39-7f78981d9dce-kube-api-access-tk5c2\") pod \"payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr\" (UID: \"28b52332-c299-4f36-9e39-7f78981d9dce\") " pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:33.270214 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.270192    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv2lk\" (UniqueName: \"kubernetes.io/projected/9fb31742-33d3-4d73-88e2-644e20a6ef4a-kube-api-access-hv2lk\") pod \"payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd\" (UID: \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:33.405918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.405837    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:33.443129 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.443024    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:33.679079 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.679049    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"]
Sep 16 14:33:33.680963 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:33:33.680936    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb31742_33d3_4d73_88e2_644e20a6ef4a.slice/crio-279309e3e54d397d61aa8e7247f04923b1694c6d690f95b3c05bbbebdb95484e WatchSource:0}: Error finding container 279309e3e54d397d61aa8e7247f04923b1694c6d690f95b3c05bbbebdb95484e: Status 404 returned error can't find the container with id 279309e3e54d397d61aa8e7247f04923b1694c6d690f95b3c05bbbebdb95484e
Sep 16 14:33:33.730961 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.730940    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"]
Sep 16 14:33:33.732709 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:33:33.732684    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b52332_c299_4f36_9e39_7f78981d9dce.slice/crio-b1aa506ef8891530e24978affc92336b8c4061a6d8135f75104e5f91e090a6a7 WatchSource:0}: Error finding container b1aa506ef8891530e24978affc92336b8c4061a6d8135f75104e5f91e090a6a7: Status 404 returned error can't find the container with id b1aa506ef8891530e24978affc92336b8c4061a6d8135f75104e5f91e090a6a7
Sep 16 14:33:33.938001 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.937896    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd" event={"ID":"9fb31742-33d3-4d73-88e2-644e20a6ef4a","Type":"ContainerStarted","Data":"14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074"}
Sep 16 14:33:33.938001 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.937961    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd" event={"ID":"9fb31742-33d3-4d73-88e2-644e20a6ef4a","Type":"ContainerStarted","Data":"279309e3e54d397d61aa8e7247f04923b1694c6d690f95b3c05bbbebdb95484e"}
Sep 16 14:33:33.938001 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.937991    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:33.939439 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.939411    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr" event={"ID":"28b52332-c299-4f36-9e39-7f78981d9dce","Type":"ContainerStarted","Data":"42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412"}
Sep 16 14:33:33.939606 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.939457    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr" event={"ID":"28b52332-c299-4f36-9e39-7f78981d9dce","Type":"ContainerStarted","Data":"b1aa506ef8891530e24978affc92336b8c4061a6d8135f75104e5f91e090a6a7"}
Sep 16 14:33:33.939606 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.939513    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:33.967078 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.967024    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd" podStartSLOduration=0.967004945 podStartE2EDuration="967.004945ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:33.964855916 +0000 UTC m=+1533.442088013" watchObservedRunningTime="2026-09-16 14:33:33.967004945 +0000 UTC m=+1533.444237027"
Sep 16 14:33:33.984888 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:33.984843    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr" podStartSLOduration=0.984828045 podStartE2EDuration="984.828045ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:33.982743362 +0000 UTC m=+1533.459975450" watchObservedRunningTime="2026-09-16 14:33:33.984828045 +0000 UTC m=+1533.462060141"
Sep 16 14:33:44.946046 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:44.946014    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:33:44.946500 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:44.946085    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:33:49.403880 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:33:49.403844    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:34:01.123373 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:01.123341    2566 scope.go:117] "RemoveContainer" containerID="2b155bd380d06af6c764a1c89ccd7d59e4e0a9630ffd87a8f205f9a1f1d5e194"
Sep 16 14:34:01.131746 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:01.131717    2566 scope.go:117] "RemoveContainer" containerID="02bd076f37e52b2d3d4e8ae915ab1002c669aaaaf2e61ae06b800450f3473b61"
Sep 16 14:34:06.896039 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:06.896000    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"]
Sep 16 14:34:06.901464 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:06.901439    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:06.905001 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:06.904974    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"sub-test-model-d31d7c-kserve-self-signed-certs\""
Sep 16 14:34:06.905163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:06.905110    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"kube-root-ca.crt\""
Sep 16 14:34:06.905163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:06.905118    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"default-dockercfg-nntwx\""
Sep 16 14:34:06.905163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:06.905147    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"openshift-service-ca.crt\""
Sep 16 14:34:06.914152 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:06.914130    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"]
Sep 16 14:34:07.068816 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.068784    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-home\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.068816 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.068818    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-model-cache\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.069035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.068839    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7lh6\" (UniqueName: \"kubernetes.io/projected/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-kube-api-access-k7lh6\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.069035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.068878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-dshm\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.069035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.068931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tls-certs\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.069035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.068968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tmp-dir\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.169980 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.169894    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tmp-dir\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.170154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.169990    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-home\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.170154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.170007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-model-cache\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.170154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.170077    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7lh6\" (UniqueName: \"kubernetes.io/projected/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-kube-api-access-k7lh6\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.170154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.170126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-dshm\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.170414 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.170191    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tls-certs\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.170414 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.170385    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tmp-dir\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.170772 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.170743    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-home\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.171079 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.170746    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-model-cache\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.172597 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.172558    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-dshm\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.172857 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.172840    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tls-certs\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.180161 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.180130    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7lh6\" (UniqueName: \"kubernetes.io/projected/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-kube-api-access-k7lh6\") pod \"sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.211604 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.211560    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:07.485484 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:07.485439    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"]
Sep 16 14:34:07.487782 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:34:07.487751    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2efe6063_2b4d_43d2_84ce_ada720c2c4d8.slice/crio-2b7f4ae0153dda6cac074d2f367c6ba54ef6490857879f44c6d8370fd946e28c WatchSource:0}: Error finding container 2b7f4ae0153dda6cac074d2f367c6ba54ef6490857879f44c6d8370fd946e28c: Status 404 returned error can't find the container with id 2b7f4ae0153dda6cac074d2f367c6ba54ef6490857879f44c6d8370fd946e28c
Sep 16 14:34:08.108911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:08.108869    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv" event={"ID":"2efe6063-2b4d-43d2-84ce-ada720c2c4d8","Type":"ContainerStarted","Data":"013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0"}
Sep 16 14:34:08.108911 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:08.108919    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv" event={"ID":"2efe6063-2b4d-43d2-84ce-ada720c2c4d8","Type":"ContainerStarted","Data":"2b7f4ae0153dda6cac074d2f367c6ba54ef6490857879f44c6d8370fd946e28c"}
Sep 16 14:34:08.128176 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:08.128128    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv" podStartSLOduration=2.128110789 podStartE2EDuration="2.128110789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:34:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:34:08.126294711 +0000 UTC m=+1567.603526807" watchObservedRunningTime="2026-09-16 14:34:08.128110789 +0000 UTC m=+1567.605342912"
Sep 16 14:34:11.611365 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:11.611331    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:34:17.212159 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:17.212123    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:17.212159 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:17.212165    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:17.224860 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:17.224832    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:18.167998 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:18.167961    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:34:37.519910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.519868    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"]
Sep 16 14:34:37.523534 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.523511    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.527444 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.527425    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9829ef2\"/\"default-dockercfg-wzqnl\""
Sep 16 14:34:37.527585 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.527546    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9829ef2\"/\"kube-root-ca.crt\""
Sep 16 14:34:37.528146 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.528132    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9829ef2\"/\"openshift-service-ca.crt\""
Sep 16 14:34:37.528277 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.528256    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9829ef2\"/\"e2e-lifecycle-model-e9829ef2-kserve-self-signed-certs\""
Sep 16 14:34:37.544423 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.544390    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"]
Sep 16 14:34:37.640652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.640621    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-home\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.640824 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.640669    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-tmp-dir\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.640824 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.640786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-dshm\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.640824 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.640818    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-model-cache\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.640934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.640836    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45af1454-d8db-4428-b276-c0616994d7bc-tls-certs\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.640934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.640905    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v556j\" (UniqueName: \"kubernetes.io/projected/45af1454-d8db-4428-b276-c0616994d7bc-kube-api-access-v556j\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.741584 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.741530    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-dshm\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.741584 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.741586    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-model-cache\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.741838 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.741693    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45af1454-d8db-4428-b276-c0616994d7bc-tls-certs\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.741838 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.741763    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v556j\" (UniqueName: \"kubernetes.io/projected/45af1454-d8db-4428-b276-c0616994d7bc-kube-api-access-v556j\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.741838 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.741793    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-home\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.741972 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.741840    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-tmp-dir\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.741972 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.741901    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-model-cache\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.742171 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.742149    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-tmp-dir\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.742217 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.742184    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-home\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.744187 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.744164    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-dshm\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.744340 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.744294    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45af1454-d8db-4428-b276-c0616994d7bc-tls-certs\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.751550 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.751528    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v556j\" (UniqueName: \"kubernetes.io/projected/45af1454-d8db-4428-b276-c0616994d7bc-kube-api-access-v556j\") pod \"e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") " pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:37.833944 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:37.833861    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:38.117988 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:38.117966    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"]
Sep 16 14:34:38.120492 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:34:38.120462    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45af1454_d8db_4428_b276_c0616994d7bc.slice/crio-86b6f768fc198013b061bcde3a473aa40bd2dba99f3ec54c51d727fdba007c2a WatchSource:0}: Error finding container 86b6f768fc198013b061bcde3a473aa40bd2dba99f3ec54c51d727fdba007c2a: Status 404 returned error can't find the container with id 86b6f768fc198013b061bcde3a473aa40bd2dba99f3ec54c51d727fdba007c2a
Sep 16 14:34:38.252969 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:38.252932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6" event={"ID":"45af1454-d8db-4428-b276-c0616994d7bc","Type":"ContainerStarted","Data":"1befd12b63994314eb19813263526bb77ea0db3972e1a4880e483d83dd7ca4e6"}
Sep 16 14:34:38.252969 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:38.252975    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6" event={"ID":"45af1454-d8db-4428-b276-c0616994d7bc","Type":"ContainerStarted","Data":"86b6f768fc198013b061bcde3a473aa40bd2dba99f3ec54c51d727fdba007c2a"}
Sep 16 14:34:38.280367 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:38.280302    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6" podStartSLOduration=1.280285972 podStartE2EDuration="1.280285972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:34:38.279054535 +0000 UTC m=+1597.756286627" watchObservedRunningTime="2026-09-16 14:34:38.280285972 +0000 UTC m=+1597.757518046"
Sep 16 14:34:42.813550 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:42.813509    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:34:47.834386 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:47.834340    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:47.834386 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:47.834395    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:47.847497 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:47.847470    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:48.314218 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:48.314189    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:34:57.116407 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:34:57.116329    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:35:01.158243 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:01.158210    2566 scope.go:117] "RemoveContainer" containerID="b052330c5bdacc0429d44fd08285415960fe0fa91501ff79b05a0cbdde9c4277"
Sep 16 14:35:01.166490 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:01.166465    2566 scope.go:117] "RemoveContainer" containerID="6de3e1915d91626753c6ed1391484369da208523d20372efafbe555ce417caeb"
Sep 16 14:35:15.890528 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:15.890494    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:35:37.354873 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:37.354833    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"]
Sep 16 14:35:38.420604 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:38.420557    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"]
Sep 16 14:35:43.005941 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:43.005900    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:35:49.971499 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:49.971464    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"]
Sep 16 14:35:49.972013 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:49.971715    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr" podUID="28b52332-c299-4f36-9e39-7f78981d9dce" containerName="payload-processing" containerID="cri-o://42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412" gracePeriod=30
Sep 16 14:35:49.978061 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:49.978005    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"]
Sep 16 14:35:49.978626 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:49.978205    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd" podUID="9fb31742-33d3-4d73-88e2-644e20a6ef4a" containerName="payload-pre-processing" containerID="cri-o://14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074" gracePeriod=30
Sep 16 14:35:50.246849 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.246826    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:35:50.249627 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.249608    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:35:50.298453 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.298427    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b52332-c299-4f36-9e39-7f78981d9dce-plugins-config-volume\") pod \"28b52332-c299-4f36-9e39-7f78981d9dce\" (UID: \"28b52332-c299-4f36-9e39-7f78981d9dce\") "
Sep 16 14:35:50.298644 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.298466    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk5c2\" (UniqueName: \"kubernetes.io/projected/28b52332-c299-4f36-9e39-7f78981d9dce-kube-api-access-tk5c2\") pod \"28b52332-c299-4f36-9e39-7f78981d9dce\" (UID: \"28b52332-c299-4f36-9e39-7f78981d9dce\") "
Sep 16 14:35:50.298644 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.298482    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv2lk\" (UniqueName: \"kubernetes.io/projected/9fb31742-33d3-4d73-88e2-644e20a6ef4a-kube-api-access-hv2lk\") pod \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\" (UID: \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\") "
Sep 16 14:35:50.298644 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.298623    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb31742-33d3-4d73-88e2-644e20a6ef4a-plugins-config-volume\") pod \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\" (UID: \"9fb31742-33d3-4d73-88e2-644e20a6ef4a\") "
Sep 16 14:35:50.298954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.298927    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28b52332-c299-4f36-9e39-7f78981d9dce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28b52332-c299-4f36-9e39-7f78981d9dce" (UID: "28b52332-c299-4f36-9e39-7f78981d9dce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:50.299067 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.298963    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb31742-33d3-4d73-88e2-644e20a6ef4a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9fb31742-33d3-4d73-88e2-644e20a6ef4a" (UID: "9fb31742-33d3-4d73-88e2-644e20a6ef4a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:50.300849 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.300827    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb31742-33d3-4d73-88e2-644e20a6ef4a-kube-api-access-hv2lk" (OuterVolumeSpecName: "kube-api-access-hv2lk") pod "9fb31742-33d3-4d73-88e2-644e20a6ef4a" (UID: "9fb31742-33d3-4d73-88e2-644e20a6ef4a"). InnerVolumeSpecName "kube-api-access-hv2lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:50.300942 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.300851    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28b52332-c299-4f36-9e39-7f78981d9dce-kube-api-access-tk5c2" (OuterVolumeSpecName: "kube-api-access-tk5c2") pod "28b52332-c299-4f36-9e39-7f78981d9dce" (UID: "28b52332-c299-4f36-9e39-7f78981d9dce"). InnerVolumeSpecName "kube-api-access-tk5c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:50.400107 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.400078    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b52332-c299-4f36-9e39-7f78981d9dce-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:50.400107 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.400102    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk5c2\" (UniqueName: \"kubernetes.io/projected/28b52332-c299-4f36-9e39-7f78981d9dce-kube-api-access-tk5c2\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:50.400107 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.400113    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv2lk\" (UniqueName: \"kubernetes.io/projected/9fb31742-33d3-4d73-88e2-644e20a6ef4a-kube-api-access-hv2lk\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:50.400308 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.400122    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb31742-33d3-4d73-88e2-644e20a6ef4a-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:50.612285 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.612203    2566 generic.go:356] "Generic (PLEG): container finished" podID="9fb31742-33d3-4d73-88e2-644e20a6ef4a" containerID="14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074" exitCode=0
Sep 16 14:35:50.612285 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.612247    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"
Sep 16 14:35:50.612526 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.612283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd" event={"ID":"9fb31742-33d3-4d73-88e2-644e20a6ef4a","Type":"ContainerDied","Data":"14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074"}
Sep 16 14:35:50.612526 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.612331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd" event={"ID":"9fb31742-33d3-4d73-88e2-644e20a6ef4a","Type":"ContainerDied","Data":"279309e3e54d397d61aa8e7247f04923b1694c6d690f95b3c05bbbebdb95484e"}
Sep 16 14:35:50.612526 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.612354    2566 scope.go:117] "RemoveContainer" containerID="14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074"
Sep 16 14:35:50.613645 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.613619    2566 generic.go:356] "Generic (PLEG): container finished" podID="28b52332-c299-4f36-9e39-7f78981d9dce" containerID="42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412" exitCode=0
Sep 16 14:35:50.613766 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.613651    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"
Sep 16 14:35:50.613766 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.613700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr" event={"ID":"28b52332-c299-4f36-9e39-7f78981d9dce","Type":"ContainerDied","Data":"42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412"}
Sep 16 14:35:50.613766 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.613751    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr" event={"ID":"28b52332-c299-4f36-9e39-7f78981d9dce","Type":"ContainerDied","Data":"b1aa506ef8891530e24978affc92336b8c4061a6d8135f75104e5f91e090a6a7"}
Sep 16 14:35:50.621993 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.621975    2566 scope.go:117] "RemoveContainer" containerID="14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074"
Sep 16 14:35:50.622228 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:35:50.622209    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074\": container with ID starting with 14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074 not found: ID does not exist" containerID="14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074"
Sep 16 14:35:50.622285 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.622236    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074"} err="failed to get container status \"14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074\": rpc error: code = NotFound desc = could not find container \"14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074\": container with ID starting with 14cebdb06181fb6443d40894cd3f65c0feadbf159712ad2c9799682db081e074 not found: ID does not exist"
Sep 16 14:35:50.622285 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.622253    2566 scope.go:117] "RemoveContainer" containerID="42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412"
Sep 16 14:35:50.630580 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.630552    2566 scope.go:117] "RemoveContainer" containerID="42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412"
Sep 16 14:35:50.630845 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:35:50.630829    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412\": container with ID starting with 42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412 not found: ID does not exist" containerID="42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412"
Sep 16 14:35:50.630893 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.630853    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412"} err="failed to get container status \"42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412\": rpc error: code = NotFound desc = could not find container \"42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412\": container with ID starting with 42095956c0d6822ae6e9f7d0a164b518ed5b226ac2fa1f5074463b5c06682412 not found: ID does not exist"
Sep 16 14:35:50.640942 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.640918    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"]
Sep 16 14:35:50.654997 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.654978    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9829ef2-f7c8588c4-2llpd"]
Sep 16 14:35:50.678150 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.678130    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"]
Sep 16 14:35:50.686187 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.686168    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9829ef2-658bc7f4b5-nfswr"]
Sep 16 14:35:50.985871 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.985841    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28b52332-c299-4f36-9e39-7f78981d9dce" path="/var/lib/kubelet/pods/28b52332-c299-4f36-9e39-7f78981d9dce/volumes"
Sep 16 14:35:50.986373 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:50.986352    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fb31742-33d3-4d73-88e2-644e20a6ef4a" path="/var/lib/kubelet/pods/9fb31742-33d3-4d73-88e2-644e20a6ef4a/volumes"
Sep 16 14:35:52.374132 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.374089    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd" podUID="007bd32d-4705-4b03-a755-7aa08ff92e0b" containerName="main" containerID="cri-o://9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c" gracePeriod=45
Sep 16 14:35:52.620049 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.620025    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:35:52.625508 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.625447    2566 generic.go:356] "Generic (PLEG): container finished" podID="007bd32d-4705-4b03-a755-7aa08ff92e0b" containerID="9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c" exitCode=0
Sep 16 14:35:52.625508 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.625474    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"
Sep 16 14:35:52.625667 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.625560    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd" event={"ID":"007bd32d-4705-4b03-a755-7aa08ff92e0b","Type":"ContainerDied","Data":"9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c"}
Sep 16 14:35:52.625667 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.625605    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd" event={"ID":"007bd32d-4705-4b03-a755-7aa08ff92e0b","Type":"ContainerDied","Data":"2f4287234b5d6a9887df72b46f7f9d3b4b3b8d4acb007c6ec3093ca22fefc29a"}
Sep 16 14:35:52.625667 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.625622    2566 scope.go:117] "RemoveContainer" containerID="9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c"
Sep 16 14:35:52.633724 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.633691    2566 scope.go:117] "RemoveContainer" containerID="9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c"
Sep 16 14:35:52.633977 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:35:52.633956    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c\": container with ID starting with 9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c not found: ID does not exist" containerID="9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c"
Sep 16 14:35:52.634050 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.633989    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c"} err="failed to get container status \"9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c\": rpc error: code = NotFound desc = could not find container \"9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c\": container with ID starting with 9e6d0bb15aba3b8e04296c8aad10bf33eebb5a4126de9c694ed77a6dbbdf9b4c not found: ID does not exist"
Sep 16 14:35:52.721072 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721042    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/007bd32d-4705-4b03-a755-7aa08ff92e0b-tls-certs\") pod \"007bd32d-4705-4b03-a755-7aa08ff92e0b\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") "
Sep 16 14:35:52.721253 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721101    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-dshm\") pod \"007bd32d-4705-4b03-a755-7aa08ff92e0b\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") "
Sep 16 14:35:52.721253 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721125    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-tmp-dir\") pod \"007bd32d-4705-4b03-a755-7aa08ff92e0b\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") "
Sep 16 14:35:52.721253 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721172    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-model-cache\") pod \"007bd32d-4705-4b03-a755-7aa08ff92e0b\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") "
Sep 16 14:35:52.721253 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721210    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swfnb\" (UniqueName: \"kubernetes.io/projected/007bd32d-4705-4b03-a755-7aa08ff92e0b-kube-api-access-swfnb\") pod \"007bd32d-4705-4b03-a755-7aa08ff92e0b\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") "
Sep 16 14:35:52.721253 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721241    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-home\") pod \"007bd32d-4705-4b03-a755-7aa08ff92e0b\" (UID: \"007bd32d-4705-4b03-a755-7aa08ff92e0b\") "
Sep 16 14:35:52.721513 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721464    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "007bd32d-4705-4b03-a755-7aa08ff92e0b" (UID: "007bd32d-4705-4b03-a755-7aa08ff92e0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:52.721513 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721475    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "007bd32d-4705-4b03-a755-7aa08ff92e0b" (UID: "007bd32d-4705-4b03-a755-7aa08ff92e0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:52.721647 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721586    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-home" (OuterVolumeSpecName: "home") pod "007bd32d-4705-4b03-a755-7aa08ff92e0b" (UID: "007bd32d-4705-4b03-a755-7aa08ff92e0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:52.721701 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721656    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:52.721701 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.721680    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:52.723375 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.723354    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-dshm" (OuterVolumeSpecName: "dshm") pod "007bd32d-4705-4b03-a755-7aa08ff92e0b" (UID: "007bd32d-4705-4b03-a755-7aa08ff92e0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:52.723533 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.723515    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/007bd32d-4705-4b03-a755-7aa08ff92e0b-kube-api-access-swfnb" (OuterVolumeSpecName: "kube-api-access-swfnb") pod "007bd32d-4705-4b03-a755-7aa08ff92e0b" (UID: "007bd32d-4705-4b03-a755-7aa08ff92e0b"). InnerVolumeSpecName "kube-api-access-swfnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:52.723768 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.723753    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/007bd32d-4705-4b03-a755-7aa08ff92e0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "007bd32d-4705-4b03-a755-7aa08ff92e0b" (UID: "007bd32d-4705-4b03-a755-7aa08ff92e0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:35:52.822988 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.822957    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/007bd32d-4705-4b03-a755-7aa08ff92e0b-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:52.822988 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.822981    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:52.822988 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.822990    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swfnb\" (UniqueName: \"kubernetes.io/projected/007bd32d-4705-4b03-a755-7aa08ff92e0b-kube-api-access-swfnb\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:52.823208 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.823000    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/007bd32d-4705-4b03-a755-7aa08ff92e0b-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:52.947919 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.947893    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"]
Sep 16 14:35:52.952348 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.952313    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/sub-test-model-c0974e-kserve-77f5d6cd87-nrrzd"]
Sep 16 14:35:52.983614 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:52.983556    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="007bd32d-4705-4b03-a755-7aa08ff92e0b" path="/var/lib/kubelet/pods/007bd32d-4705-4b03-a755-7aa08ff92e0b/volumes"
Sep 16 14:35:53.567816 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.567767    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv" podUID="2efe6063-2b4d-43d2-84ce-ada720c2c4d8" containerName="main" containerID="cri-o://013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0" gracePeriod=45
Sep 16 14:35:53.818505 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.818445    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:35:53.933421 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933373    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-model-cache\") pod \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") "
Sep 16 14:35:53.933421 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933437    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tls-certs\") pod \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") "
Sep 16 14:35:53.933732 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933453    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tmp-dir\") pod \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") "
Sep 16 14:35:53.933732 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933473    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-home\") pod \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") "
Sep 16 14:35:53.933732 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933511    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-dshm\") pod \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") "
Sep 16 14:35:53.933732 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933545    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7lh6\" (UniqueName: \"kubernetes.io/projected/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-kube-api-access-k7lh6\") pod \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\" (UID: \"2efe6063-2b4d-43d2-84ce-ada720c2c4d8\") "
Sep 16 14:35:53.933943 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933756    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "2efe6063-2b4d-43d2-84ce-ada720c2c4d8" (UID: "2efe6063-2b4d-43d2-84ce-ada720c2c4d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:53.933943 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933767    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2efe6063-2b4d-43d2-84ce-ada720c2c4d8" (UID: "2efe6063-2b4d-43d2-84ce-ada720c2c4d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:53.933943 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933871    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:53.933943 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933879    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-home" (OuterVolumeSpecName: "home") pod "2efe6063-2b4d-43d2-84ce-ada720c2c4d8" (UID: "2efe6063-2b4d-43d2-84ce-ada720c2c4d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:53.933943 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.933889    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:53.936028 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.936001    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-dshm" (OuterVolumeSpecName: "dshm") pod "2efe6063-2b4d-43d2-84ce-ada720c2c4d8" (UID: "2efe6063-2b4d-43d2-84ce-ada720c2c4d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:53.936028 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.936000    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2efe6063-2b4d-43d2-84ce-ada720c2c4d8" (UID: "2efe6063-2b4d-43d2-84ce-ada720c2c4d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:35:53.936175 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:53.936040    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-kube-api-access-k7lh6" (OuterVolumeSpecName: "kube-api-access-k7lh6") pod "2efe6063-2b4d-43d2-84ce-ada720c2c4d8" (UID: "2efe6063-2b4d-43d2-84ce-ada720c2c4d8"). InnerVolumeSpecName "kube-api-access-k7lh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:54.035160 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.035127    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:54.035160 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.035155    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7lh6\" (UniqueName: \"kubernetes.io/projected/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-kube-api-access-k7lh6\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:54.035160 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.035165    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:54.035394 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.035175    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2efe6063-2b4d-43d2-84ce-ada720c2c4d8-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:35:54.636704 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.636667    2566 generic.go:356] "Generic (PLEG): container finished" podID="2efe6063-2b4d-43d2-84ce-ada720c2c4d8" containerID="013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0" exitCode=0
Sep 16 14:35:54.637086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.636720    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"
Sep 16 14:35:54.637086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.636755    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv" event={"ID":"2efe6063-2b4d-43d2-84ce-ada720c2c4d8","Type":"ContainerDied","Data":"013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0"}
Sep 16 14:35:54.637086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.636800    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv" event={"ID":"2efe6063-2b4d-43d2-84ce-ada720c2c4d8","Type":"ContainerDied","Data":"2b7f4ae0153dda6cac074d2f367c6ba54ef6490857879f44c6d8370fd946e28c"}
Sep 16 14:35:54.637086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.636817    2566 scope.go:117] "RemoveContainer" containerID="013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0"
Sep 16 14:35:54.645629 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.645608    2566 scope.go:117] "RemoveContainer" containerID="013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0"
Sep 16 14:35:54.645886 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:35:54.645866    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0\": container with ID starting with 013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0 not found: ID does not exist" containerID="013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0"
Sep 16 14:35:54.645960 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.645894    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0"} err="failed to get container status \"013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0\": rpc error: code = NotFound desc = could not find container \"013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0\": container with ID starting with 013b1d9aea82fdd3f33bf083b699a701fef4ddf2b89d7ab323954f4271415eb0 not found: ID does not exist"
Sep 16 14:35:54.660451 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.660430    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"]
Sep 16 14:35:54.668964 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.668945    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/sub-test-model-d31d7c-kserve-6554c8bdcf-2f7rv"]
Sep 16 14:35:54.984054 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:54.984023    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2efe6063-2b4d-43d2-84ce-ada720c2c4d8" path="/var/lib/kubelet/pods/2efe6063-2b4d-43d2-84ce-ada720c2c4d8/volumes"
Sep 16 14:35:57.299542 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.299509    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"]
Sep 16 14:35:57.300051 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300030    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fb31742-33d3-4d73-88e2-644e20a6ef4a" containerName="payload-pre-processing"
Sep 16 14:35:57.300195 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300055    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fb31742-33d3-4d73-88e2-644e20a6ef4a" containerName="payload-pre-processing"
Sep 16 14:35:57.300195 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300069    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28b52332-c299-4f36-9e39-7f78981d9dce" containerName="payload-processing"
Sep 16 14:35:57.300195 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300078    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="28b52332-c299-4f36-9e39-7f78981d9dce" containerName="payload-processing"
Sep 16 14:35:57.300195 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300109    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="007bd32d-4705-4b03-a755-7aa08ff92e0b" containerName="main"
Sep 16 14:35:57.300195 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300119    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="007bd32d-4705-4b03-a755-7aa08ff92e0b" containerName="main"
Sep 16 14:35:57.300195 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300135    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2efe6063-2b4d-43d2-84ce-ada720c2c4d8" containerName="main"
Sep 16 14:35:57.300195 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300144    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2efe6063-2b4d-43d2-84ce-ada720c2c4d8" containerName="main"
Sep 16 14:35:57.300522 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300249    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2efe6063-2b4d-43d2-84ce-ada720c2c4d8" containerName="main"
Sep 16 14:35:57.300522 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300261    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="007bd32d-4705-4b03-a755-7aa08ff92e0b" containerName="main"
Sep 16 14:35:57.300522 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300273    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="28b52332-c299-4f36-9e39-7f78981d9dce" containerName="payload-processing"
Sep 16 14:35:57.300522 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.300283    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fb31742-33d3-4d73-88e2-644e20a6ef4a" containerName="payload-pre-processing"
Sep 16 14:35:57.304734 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.304712    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.307488 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.307467    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-c0974e\"/\"kube-root-ca.crt\""
Sep 16 14:35:57.308194 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.308174    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-c0974e\"/\"rate-test-model-c0974e-kserve-self-signed-certs\""
Sep 16 14:35:57.308301 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.308193    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-c0974e\"/\"default-dockercfg-cm94w\""
Sep 16 14:35:57.308301 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.308275    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-c0974e\"/\"openshift-service-ca.crt\""
Sep 16 14:35:57.314307 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.314283    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"]
Sep 16 14:35:57.363059 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.363000    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672f934-2927-41e4-bafb-d78edb6b5404-tls-certs\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.363059 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.363053    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-home\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.363249 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.363128    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-tmp-dir\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.363249 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.363220    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-model-cache\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.363368 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.363253    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w7fr\" (UniqueName: \"kubernetes.io/projected/5672f934-2927-41e4-bafb-d78edb6b5404-kube-api-access-7w7fr\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.363368 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.363301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-dshm\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.463924 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.463895    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-tmp-dir\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.464086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.463936    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-model-cache\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.464086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.463953    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w7fr\" (UniqueName: \"kubernetes.io/projected/5672f934-2927-41e4-bafb-d78edb6b5404-kube-api-access-7w7fr\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.464086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.463974    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-dshm\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.464236 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.464092    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672f934-2927-41e4-bafb-d78edb6b5404-tls-certs\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.464236 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.464142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-home\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.464414 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.464394    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-tmp-dir\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.464468 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.464418    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-model-cache\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.464514 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.464481    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-home\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.466361 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.466334    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-dshm\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.466623 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.466608    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672f934-2927-41e4-bafb-d78edb6b5404-tls-certs\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.474722 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.474699    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w7fr\" (UniqueName: \"kubernetes.io/projected/5672f934-2927-41e4-bafb-d78edb6b5404-kube-api-access-7w7fr\") pod \"rate-test-model-c0974e-kserve-7974598b6c-bc8bz\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") " pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.615364 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.615286    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:35:57.902993 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:57.902915    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"]
Sep 16 14:35:57.909873 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:35:57.909839    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5672f934_2927_41e4_bafb_d78edb6b5404.slice/crio-c42c64ad0fecc941a00c56ae5b77f286c84558523b66b1335a3d53227e91a0d4 WatchSource:0}: Error finding container c42c64ad0fecc941a00c56ae5b77f286c84558523b66b1335a3d53227e91a0d4: Status 404 returned error can't find the container with id c42c64ad0fecc941a00c56ae5b77f286c84558523b66b1335a3d53227e91a0d4
Sep 16 14:35:58.070550 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:58.070513    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:35:58.510401 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:58.510365    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"]
Sep 16 14:35:58.658205 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:58.658167    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz" event={"ID":"5672f934-2927-41e4-bafb-d78edb6b5404","Type":"ContainerStarted","Data":"2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6"}
Sep 16 14:35:58.658205 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:58.658207    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz" event={"ID":"5672f934-2927-41e4-bafb-d78edb6b5404","Type":"ContainerStarted","Data":"c42c64ad0fecc941a00c56ae5b77f286c84558523b66b1335a3d53227e91a0d4"}
Sep 16 14:35:58.681593 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:35:58.678497    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz" podStartSLOduration=1.67847833 podStartE2EDuration="1.67847833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:35:58.676523895 +0000 UTC m=+1678.153755984" watchObservedRunningTime="2026-09-16 14:35:58.67847833 +0000 UTC m=+1678.155710407"
Sep 16 14:36:01.187774 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:01.187742    2566 scope.go:117] "RemoveContainer" containerID="e074ecae29b9a6ac4e8ee5b80321792488c91a0b36f94f204cd0df7d4d39fb95"
Sep 16 14:36:07.615655 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:07.615608    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:36:07.616108 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:07.615686    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:36:07.628495 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:07.628471    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:36:07.724228 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:07.724200    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:36:08.202698 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:08.202666    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:36:13.530016 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.529971    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6" podUID="45af1454-d8db-4428-b276-c0616994d7bc" containerName="main" containerID="cri-o://1befd12b63994314eb19813263526bb77ea0db3972e1a4880e483d83dd7ca4e6" gracePeriod=45
Sep 16 14:36:13.741954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.741922    2566 generic.go:356] "Generic (PLEG): container finished" podID="45af1454-d8db-4428-b276-c0616994d7bc" containerID="1befd12b63994314eb19813263526bb77ea0db3972e1a4880e483d83dd7ca4e6" exitCode=0
Sep 16 14:36:13.742126 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.742000    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6" event={"ID":"45af1454-d8db-4428-b276-c0616994d7bc","Type":"ContainerDied","Data":"1befd12b63994314eb19813263526bb77ea0db3972e1a4880e483d83dd7ca4e6"}
Sep 16 14:36:13.770838 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.770813    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:36:13.910619 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.910523    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-model-cache\") pod \"45af1454-d8db-4428-b276-c0616994d7bc\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") "
Sep 16 14:36:13.910619 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.910586    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v556j\" (UniqueName: \"kubernetes.io/projected/45af1454-d8db-4428-b276-c0616994d7bc-kube-api-access-v556j\") pod \"45af1454-d8db-4428-b276-c0616994d7bc\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") "
Sep 16 14:36:13.910837 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.910639    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-home\") pod \"45af1454-d8db-4428-b276-c0616994d7bc\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") "
Sep 16 14:36:13.910837 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.910662    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-dshm\") pod \"45af1454-d8db-4428-b276-c0616994d7bc\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") "
Sep 16 14:36:13.910837 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.910693    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-tmp-dir\") pod \"45af1454-d8db-4428-b276-c0616994d7bc\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") "
Sep 16 14:36:13.910837 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.910754    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45af1454-d8db-4428-b276-c0616994d7bc-tls-certs\") pod \"45af1454-d8db-4428-b276-c0616994d7bc\" (UID: \"45af1454-d8db-4428-b276-c0616994d7bc\") "
Sep 16 14:36:13.911037 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.910830    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "45af1454-d8db-4428-b276-c0616994d7bc" (UID: "45af1454-d8db-4428-b276-c0616994d7bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:36:13.911037 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.910859    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-home" (OuterVolumeSpecName: "home") pod "45af1454-d8db-4428-b276-c0616994d7bc" (UID: "45af1454-d8db-4428-b276-c0616994d7bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:36:13.911131 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.911064    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:36:13.911131 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.911080    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:36:13.911131 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.911089    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "45af1454-d8db-4428-b276-c0616994d7bc" (UID: "45af1454-d8db-4428-b276-c0616994d7bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:36:13.913045 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.913012    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-dshm" (OuterVolumeSpecName: "dshm") pod "45af1454-d8db-4428-b276-c0616994d7bc" (UID: "45af1454-d8db-4428-b276-c0616994d7bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:36:13.913142 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.913058    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45af1454-d8db-4428-b276-c0616994d7bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "45af1454-d8db-4428-b276-c0616994d7bc" (UID: "45af1454-d8db-4428-b276-c0616994d7bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:36:13.913142 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:13.913070    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45af1454-d8db-4428-b276-c0616994d7bc-kube-api-access-v556j" (OuterVolumeSpecName: "kube-api-access-v556j") pod "45af1454-d8db-4428-b276-c0616994d7bc" (UID: "45af1454-d8db-4428-b276-c0616994d7bc"). InnerVolumeSpecName "kube-api-access-v556j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:36:14.011684 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.011649    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:36:14.011684 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.011679    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45af1454-d8db-4428-b276-c0616994d7bc-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:36:14.011684 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.011688    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v556j\" (UniqueName: \"kubernetes.io/projected/45af1454-d8db-4428-b276-c0616994d7bc-kube-api-access-v556j\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:36:14.011910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.011697    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45af1454-d8db-4428-b276-c0616994d7bc-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:36:14.748055 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.748019    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6" event={"ID":"45af1454-d8db-4428-b276-c0616994d7bc","Type":"ContainerDied","Data":"86b6f768fc198013b061bcde3a473aa40bd2dba99f3ec54c51d727fdba007c2a"}
Sep 16 14:36:14.748055 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.748050    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"
Sep 16 14:36:14.748529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.748068    2566 scope.go:117] "RemoveContainer" containerID="1befd12b63994314eb19813263526bb77ea0db3972e1a4880e483d83dd7ca4e6"
Sep 16 14:36:14.774651 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.774624    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"]
Sep 16 14:36:14.778825 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.778798    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e9829ef2/e2e-lifecycle-model-e9829ef2-kserve-7bcb55655f-6rbk6"]
Sep 16 14:36:14.983553 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:14.983518    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45af1454-d8db-4428-b276-c0616994d7bc" path="/var/lib/kubelet/pods/45af1454-d8db-4428-b276-c0616994d7bc/volumes"
Sep 16 14:36:47.493772 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.493734    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"]
Sep 16 14:36:47.494256 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.494108    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45af1454-d8db-4428-b276-c0616994d7bc" containerName="main"
Sep 16 14:36:47.494256 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.494120    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="45af1454-d8db-4428-b276-c0616994d7bc" containerName="main"
Sep 16 14:36:47.494256 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.494191    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="45af1454-d8db-4428-b276-c0616994d7bc" containerName="main"
Sep 16 14:36:47.497161 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.497144    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.504245 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.504217    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"kube-root-ca.crt\""
Sep 16 14:36:47.505010 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.504992    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"openshift-service-ca.crt\""
Sep 16 14:36:47.505116 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.505065    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"default-dockercfg-nntwx\""
Sep 16 14:36:47.505116 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.505093    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"rate-test-model-d31d7c-kserve-self-signed-certs\""
Sep 16 14:36:47.525163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.525133    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"]
Sep 16 14:36:47.595887 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.595849    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-dshm\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.595887 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.595888    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4764\" (UniqueName: \"kubernetes.io/projected/687987a9-30ff-43da-a991-2984034d34e1-kube-api-access-f4764\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.596125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.595979    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/687987a9-30ff-43da-a991-2984034d34e1-tls-certs\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.596125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.596022    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-model-cache\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.596125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.596065    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-tmp-dir\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.596125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.596111    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-home\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.696766 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.696728    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-tmp-dir\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.696963 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.696779    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-home\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.696963 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.696802    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-dshm\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.696963 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.696916    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4764\" (UniqueName: \"kubernetes.io/projected/687987a9-30ff-43da-a991-2984034d34e1-kube-api-access-f4764\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.697129 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.696975    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/687987a9-30ff-43da-a991-2984034d34e1-tls-certs\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.697188 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.697138    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-model-cache\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.697246 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.697189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-home\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.697302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.697253    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-tmp-dir\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.697518 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.697497    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-model-cache\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.699237 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.699214    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-dshm\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.699659 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.699639    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/687987a9-30ff-43da-a991-2984034d34e1-tls-certs\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.705614 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.705560    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4764\" (UniqueName: \"kubernetes.io/projected/687987a9-30ff-43da-a991-2984034d34e1-kube-api-access-f4764\") pod \"rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:47.806085 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:47.806008    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:48.082178 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:48.082151    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"]
Sep 16 14:36:48.083991 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:36:48.083962    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod687987a9_30ff_43da_a991_2984034d34e1.slice/crio-140f83522b423c3151324340e28540b9b2ad646f2a59d39c16c059ab906b32aa WatchSource:0}: Error finding container 140f83522b423c3151324340e28540b9b2ad646f2a59d39c16c059ab906b32aa: Status 404 returned error can't find the container with id 140f83522b423c3151324340e28540b9b2ad646f2a59d39c16c059ab906b32aa
Sep 16 14:36:48.911945 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:48.911904    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt" event={"ID":"687987a9-30ff-43da-a991-2984034d34e1","Type":"ContainerStarted","Data":"3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39"}
Sep 16 14:36:48.911945 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:48.911946    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt" event={"ID":"687987a9-30ff-43da-a991-2984034d34e1","Type":"ContainerStarted","Data":"140f83522b423c3151324340e28540b9b2ad646f2a59d39c16c059ab906b32aa"}
Sep 16 14:36:48.930782 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:48.930731    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt" podStartSLOduration=1.9307165039999998 podStartE2EDuration="1.930716504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:36:48.929963371 +0000 UTC m=+1728.407195446" watchObservedRunningTime="2026-09-16 14:36:48.930716504 +0000 UTC m=+1728.407948578"
Sep 16 14:36:52.003398 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:52.003361    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:36:57.806685 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:57.806647    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:57.806685 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:57.806688    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:57.819207 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:57.819176    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:36:57.967545 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:36:57.967513    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:37:05.595777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:05.595741    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:37:23.396929 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:23.396894    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:37:38.098330 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:38.098295    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:37:42.507512 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:42.507477    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"]
Sep 16 14:37:43.491850 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:43.491813    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"]
Sep 16 14:37:50.932135 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:50.932098    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"]
Sep 16 14:37:50.937465 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:50.937431    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:37:50.941297 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:50.941274    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-363d71\""
Sep 16 14:37:50.949591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:50.949547    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"]
Sep 16 14:37:51.058508 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:51.058470    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/946bc9b8-228d-4203-835e-634c615c7a03-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht\" (UID: \"946bc9b8-228d-4203-835e-634c615c7a03\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:37:51.058742 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:51.058641    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74n69\" (UniqueName: \"kubernetes.io/projected/946bc9b8-228d-4203-835e-634c615c7a03-kube-api-access-74n69\") pod \"payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht\" (UID: \"946bc9b8-228d-4203-835e-634c615c7a03\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:37:51.159977 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:51.159940    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/946bc9b8-228d-4203-835e-634c615c7a03-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht\" (UID: \"946bc9b8-228d-4203-835e-634c615c7a03\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:37:51.160149 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:51.160027    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74n69\" (UniqueName: \"kubernetes.io/projected/946bc9b8-228d-4203-835e-634c615c7a03-kube-api-access-74n69\") pod \"payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht\" (UID: \"946bc9b8-228d-4203-835e-634c615c7a03\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:37:51.160662 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:51.160644    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/946bc9b8-228d-4203-835e-634c615c7a03-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht\" (UID: \"946bc9b8-228d-4203-835e-634c615c7a03\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:37:51.169367 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:51.169345    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74n69\" (UniqueName: \"kubernetes.io/projected/946bc9b8-228d-4203-835e-634c615c7a03-kube-api-access-74n69\") pod \"payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht\" (UID: \"946bc9b8-228d-4203-835e-634c615c7a03\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:37:51.247647 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:51.247620    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:37:51.525602 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:51.525560    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"]
Sep 16 14:37:51.527317 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:37:51.527288    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946bc9b8_228d_4203_835e_634c615c7a03.slice/crio-ecfababa05c97ce0c7ce1ed47ffd919187f5cf181f7370a61f88020f58ece43b WatchSource:0}: Error finding container ecfababa05c97ce0c7ce1ed47ffd919187f5cf181f7370a61f88020f58ece43b: Status 404 returned error can't find the container with id ecfababa05c97ce0c7ce1ed47ffd919187f5cf181f7370a61f88020f58ece43b
Sep 16 14:37:52.208081 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:52.208045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht" event={"ID":"946bc9b8-228d-4203-835e-634c615c7a03","Type":"ContainerStarted","Data":"ba20ff29447b845372683deef1ea550097da148a7d6a915b2c948b03d51f98d0"}
Sep 16 14:37:52.208482 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:52.208110    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht" event={"ID":"946bc9b8-228d-4203-835e-634c615c7a03","Type":"ContainerStarted","Data":"ecfababa05c97ce0c7ce1ed47ffd919187f5cf181f7370a61f88020f58ece43b"}
Sep 16 14:37:52.208482 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:52.208139    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:37:52.226769 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:52.226717    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht" podStartSLOduration=2.226698551 podStartE2EDuration="2.226698551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:37:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:37:52.225808619 +0000 UTC m=+1791.703040704" watchObservedRunningTime="2026-09-16 14:37:52.226698551 +0000 UTC m=+1791.703930627"
Sep 16 14:37:55.605908 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:55.605871    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:37:57.526752 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.526710    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz" podUID="5672f934-2927-41e4-bafb-d78edb6b5404" containerName="main" containerID="cri-o://2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6" gracePeriod=45
Sep 16 14:37:57.773267 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.773242    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:37:57.918868 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.918776    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-tmp-dir\") pod \"5672f934-2927-41e4-bafb-d78edb6b5404\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") "
Sep 16 14:37:57.918868 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.918833    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-dshm\") pod \"5672f934-2927-41e4-bafb-d78edb6b5404\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") "
Sep 16 14:37:57.919068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.918880    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-home\") pod \"5672f934-2927-41e4-bafb-d78edb6b5404\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") "
Sep 16 14:37:57.919068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.918909    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-model-cache\") pod \"5672f934-2927-41e4-bafb-d78edb6b5404\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") "
Sep 16 14:37:57.919068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.918958    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672f934-2927-41e4-bafb-d78edb6b5404-tls-certs\") pod \"5672f934-2927-41e4-bafb-d78edb6b5404\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") "
Sep 16 14:37:57.919068 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.918983    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w7fr\" (UniqueName: \"kubernetes.io/projected/5672f934-2927-41e4-bafb-d78edb6b5404-kube-api-access-7w7fr\") pod \"5672f934-2927-41e4-bafb-d78edb6b5404\" (UID: \"5672f934-2927-41e4-bafb-d78edb6b5404\") "
Sep 16 14:37:57.919245 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.919064    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5672f934-2927-41e4-bafb-d78edb6b5404" (UID: "5672f934-2927-41e4-bafb-d78edb6b5404"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:57.919245 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.919148    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-home" (OuterVolumeSpecName: "home") pod "5672f934-2927-41e4-bafb-d78edb6b5404" (UID: "5672f934-2927-41e4-bafb-d78edb6b5404"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:57.919245 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.919220    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-model-cache" (OuterVolumeSpecName: "model-cache") pod "5672f934-2927-41e4-bafb-d78edb6b5404" (UID: "5672f934-2927-41e4-bafb-d78edb6b5404"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:57.919398 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.919252    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:57.919398 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.919267    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:57.921231 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.921200    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-dshm" (OuterVolumeSpecName: "dshm") pod "5672f934-2927-41e4-bafb-d78edb6b5404" (UID: "5672f934-2927-41e4-bafb-d78edb6b5404"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:57.921231 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.921200    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5672f934-2927-41e4-bafb-d78edb6b5404-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5672f934-2927-41e4-bafb-d78edb6b5404" (UID: "5672f934-2927-41e4-bafb-d78edb6b5404"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:37:57.921375 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:57.921224    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5672f934-2927-41e4-bafb-d78edb6b5404-kube-api-access-7w7fr" (OuterVolumeSpecName: "kube-api-access-7w7fr") pod "5672f934-2927-41e4-bafb-d78edb6b5404" (UID: "5672f934-2927-41e4-bafb-d78edb6b5404"). InnerVolumeSpecName "kube-api-access-7w7fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:58.019684 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.019654    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5672f934-2927-41e4-bafb-d78edb6b5404-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.019684 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.019677    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w7fr\" (UniqueName: \"kubernetes.io/projected/5672f934-2927-41e4-bafb-d78edb6b5404-kube-api-access-7w7fr\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.019684 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.019687    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.019910 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.019697    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5672f934-2927-41e4-bafb-d78edb6b5404-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.242829 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.242796    2566 generic.go:356] "Generic (PLEG): container finished" podID="5672f934-2927-41e4-bafb-d78edb6b5404" containerID="2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6" exitCode=0
Sep 16 14:37:58.242999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.242843    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"
Sep 16 14:37:58.242999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.242856    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz" event={"ID":"5672f934-2927-41e4-bafb-d78edb6b5404","Type":"ContainerDied","Data":"2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6"}
Sep 16 14:37:58.242999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.242887    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz" event={"ID":"5672f934-2927-41e4-bafb-d78edb6b5404","Type":"ContainerDied","Data":"c42c64ad0fecc941a00c56ae5b77f286c84558523b66b1335a3d53227e91a0d4"}
Sep 16 14:37:58.242999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.242904    2566 scope.go:117] "RemoveContainer" containerID="2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6"
Sep 16 14:37:58.251926 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.251911    2566 scope.go:117] "RemoveContainer" containerID="2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6"
Sep 16 14:37:58.252176 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:37:58.252156    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6\": container with ID starting with 2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6 not found: ID does not exist" containerID="2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6"
Sep 16 14:37:58.252220 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.252185    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6"} err="failed to get container status \"2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6\": rpc error: code = NotFound desc = could not find container \"2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6\": container with ID starting with 2d7edc825573bd0c70d4ab77147737c3d5f4c1235f2f2edccdeae1022c5c3fb6 not found: ID does not exist"
Sep 16 14:37:58.264373 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.264346    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"]
Sep 16 14:37:58.269134 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.269113    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz"]
Sep 16 14:37:58.512185 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.512093    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt" podUID="687987a9-30ff-43da-a991-2984034d34e1" containerName="main" containerID="cri-o://3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39" gracePeriod=45
Sep 16 14:37:58.763845 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.763791    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:37:58.927405 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927374    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-model-cache\") pod \"687987a9-30ff-43da-a991-2984034d34e1\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") "
Sep 16 14:37:58.927619 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927412    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4764\" (UniqueName: \"kubernetes.io/projected/687987a9-30ff-43da-a991-2984034d34e1-kube-api-access-f4764\") pod \"687987a9-30ff-43da-a991-2984034d34e1\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") "
Sep 16 14:37:58.927619 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927439    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-dshm\") pod \"687987a9-30ff-43da-a991-2984034d34e1\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") "
Sep 16 14:37:58.927619 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927482    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-tmp-dir\") pod \"687987a9-30ff-43da-a991-2984034d34e1\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") "
Sep 16 14:37:58.927619 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927500    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-home\") pod \"687987a9-30ff-43da-a991-2984034d34e1\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") "
Sep 16 14:37:58.927619 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927544    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/687987a9-30ff-43da-a991-2984034d34e1-tls-certs\") pod \"687987a9-30ff-43da-a991-2984034d34e1\" (UID: \"687987a9-30ff-43da-a991-2984034d34e1\") "
Sep 16 14:37:58.927887 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927656    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-model-cache" (OuterVolumeSpecName: "model-cache") pod "687987a9-30ff-43da-a991-2984034d34e1" (UID: "687987a9-30ff-43da-a991-2984034d34e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:58.927887 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927765    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-home" (OuterVolumeSpecName: "home") pod "687987a9-30ff-43da-a991-2984034d34e1" (UID: "687987a9-30ff-43da-a991-2984034d34e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:58.927887 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927789    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "687987a9-30ff-43da-a991-2984034d34e1" (UID: "687987a9-30ff-43da-a991-2984034d34e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:58.928045 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927913    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.928045 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927928    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.928045 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.927938    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.929759 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.929735    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-dshm" (OuterVolumeSpecName: "dshm") pod "687987a9-30ff-43da-a991-2984034d34e1" (UID: "687987a9-30ff-43da-a991-2984034d34e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:58.929847 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.929763    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/687987a9-30ff-43da-a991-2984034d34e1-kube-api-access-f4764" (OuterVolumeSpecName: "kube-api-access-f4764") pod "687987a9-30ff-43da-a991-2984034d34e1" (UID: "687987a9-30ff-43da-a991-2984034d34e1"). InnerVolumeSpecName "kube-api-access-f4764". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:58.929847 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.929771    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/687987a9-30ff-43da-a991-2984034d34e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "687987a9-30ff-43da-a991-2984034d34e1" (UID: "687987a9-30ff-43da-a991-2984034d34e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:37:58.985708 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:58.985677    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5672f934-2927-41e4-bafb-d78edb6b5404" path="/var/lib/kubelet/pods/5672f934-2927-41e4-bafb-d78edb6b5404/volumes"
Sep 16 14:37:59.028884 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.028804    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/687987a9-30ff-43da-a991-2984034d34e1-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:59.028884 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.028830    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4764\" (UniqueName: \"kubernetes.io/projected/687987a9-30ff-43da-a991-2984034d34e1-kube-api-access-f4764\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:59.028884 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.028840    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/687987a9-30ff-43da-a991-2984034d34e1-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:37:59.248997 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.248961    2566 generic.go:356] "Generic (PLEG): container finished" podID="687987a9-30ff-43da-a991-2984034d34e1" containerID="3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39" exitCode=0
Sep 16 14:37:59.248997 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.248996    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"
Sep 16 14:37:59.249210 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.249054    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt" event={"ID":"687987a9-30ff-43da-a991-2984034d34e1","Type":"ContainerDied","Data":"3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39"}
Sep 16 14:37:59.249210 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.249095    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt" event={"ID":"687987a9-30ff-43da-a991-2984034d34e1","Type":"ContainerDied","Data":"140f83522b423c3151324340e28540b9b2ad646f2a59d39c16c059ab906b32aa"}
Sep 16 14:37:59.249210 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.249117    2566 scope.go:117] "RemoveContainer" containerID="3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39"
Sep 16 14:37:59.257580 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.257548    2566 scope.go:117] "RemoveContainer" containerID="3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39"
Sep 16 14:37:59.257933 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:37:59.257914    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39\": container with ID starting with 3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39 not found: ID does not exist" containerID="3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39"
Sep 16 14:37:59.257984 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.257940    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39"} err="failed to get container status \"3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39\": rpc error: code = NotFound desc = could not find container \"3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39\": container with ID starting with 3c512eb71c08e09e5fc498d117a7bf855c67608965455fec4bc116627eea5d39 not found: ID does not exist"
Sep 16 14:37:59.270517 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.270487    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"]
Sep 16 14:37:59.274531 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:37:59.274511    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/rate-test-model-d31d7c-kserve-5b7bc975ff-bxhlt"]
Sep 16 14:38:00.987896 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:00.987849    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="687987a9-30ff-43da-a991-2984034d34e1" path="/var/lib/kubelet/pods/687987a9-30ff-43da-a991-2984034d34e1/volumes"
Sep 16 14:38:02.714810 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:02.714755    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w2-c0974e/rate-test-model-c0974e-kserve-7974598b6c-bc8bz" podUID="5672f934-2927-41e4-bafb-d78edb6b5404" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Sep 16 14:38:03.216767 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:03.216733    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:38:04.730992 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.730958    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"]
Sep 16 14:38:04.731389 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.731311    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5672f934-2927-41e4-bafb-d78edb6b5404" containerName="main"
Sep 16 14:38:04.731389 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.731324    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5672f934-2927-41e4-bafb-d78edb6b5404" containerName="main"
Sep 16 14:38:04.731389 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.731340    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="687987a9-30ff-43da-a991-2984034d34e1" containerName="main"
Sep 16 14:38:04.731389 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.731346    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="687987a9-30ff-43da-a991-2984034d34e1" containerName="main"
Sep 16 14:38:04.731533 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.731408    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="687987a9-30ff-43da-a991-2984034d34e1" containerName="main"
Sep 16 14:38:04.731533 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.731417    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5672f934-2927-41e4-bafb-d78edb6b5404" containerName="main"
Sep 16 14:38:04.734125 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.734109    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:04.736448 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.736424    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-a2fada\""
Sep 16 14:38:04.736448 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.736434    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-a2fada-dockercfg-26nf7\""
Sep 16 14:38:04.744122 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.744097    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"]
Sep 16 14:38:04.764450 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.764425    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"]
Sep 16 14:38:04.767699 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.767683    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:04.776416 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.776391    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a81997-f339-4867-b807-ff104a37cc71-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw\" (UID: \"65a81997-f339-4867-b807-ff104a37cc71\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:04.776535 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.776474    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h84hq\" (UniqueName: \"kubernetes.io/projected/844a58b1-d795-4dc2-9673-546f4e4f9bf5-kube-api-access-h84hq\") pod \"payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq\" (UID: \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:04.776535 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.776505    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a58b1-d795-4dc2-9673-546f4e4f9bf5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq\" (UID: \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:04.776685 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.776533    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65bbm\" (UniqueName: \"kubernetes.io/projected/65a81997-f339-4867-b807-ff104a37cc71-kube-api-access-65bbm\") pod \"payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw\" (UID: \"65a81997-f339-4867-b807-ff104a37cc71\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:04.777067 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.777047    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"]
Sep 16 14:38:04.877086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.877049    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65bbm\" (UniqueName: \"kubernetes.io/projected/65a81997-f339-4867-b807-ff104a37cc71-kube-api-access-65bbm\") pod \"payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw\" (UID: \"65a81997-f339-4867-b807-ff104a37cc71\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:04.877277 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.877097    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a81997-f339-4867-b807-ff104a37cc71-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw\" (UID: \"65a81997-f339-4867-b807-ff104a37cc71\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:04.877277 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.877154    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h84hq\" (UniqueName: \"kubernetes.io/projected/844a58b1-d795-4dc2-9673-546f4e4f9bf5-kube-api-access-h84hq\") pod \"payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq\" (UID: \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:04.877277 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.877171    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a58b1-d795-4dc2-9673-546f4e4f9bf5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq\" (UID: \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:04.877906 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.877887    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a58b1-d795-4dc2-9673-546f4e4f9bf5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq\" (UID: \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:04.877906 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.877898    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a81997-f339-4867-b807-ff104a37cc71-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw\" (UID: \"65a81997-f339-4867-b807-ff104a37cc71\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:04.885767 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.885740    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65bbm\" (UniqueName: \"kubernetes.io/projected/65a81997-f339-4867-b807-ff104a37cc71-kube-api-access-65bbm\") pod \"payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw\" (UID: \"65a81997-f339-4867-b807-ff104a37cc71\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:04.885874 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:04.885788    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h84hq\" (UniqueName: \"kubernetes.io/projected/844a58b1-d795-4dc2-9673-546f4e4f9bf5-kube-api-access-h84hq\") pod \"payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq\" (UID: \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:05.045617 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:05.045506    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:05.078498 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:05.078463    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:05.321725 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:05.321699    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"]
Sep 16 14:38:05.323736 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:38:05.323708    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844a58b1_d795_4dc2_9673_546f4e4f9bf5.slice/crio-17ab824d148e184fe49337f58b4b428b86221f0b9631812c79f477bfa4d03515 WatchSource:0}: Error finding container 17ab824d148e184fe49337f58b4b428b86221f0b9631812c79f477bfa4d03515: Status 404 returned error can't find the container with id 17ab824d148e184fe49337f58b4b428b86221f0b9631812c79f477bfa4d03515
Sep 16 14:38:05.357423 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:05.357333    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"]
Sep 16 14:38:05.359520 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:38:05.359493    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a81997_f339_4867_b807_ff104a37cc71.slice/crio-2cb5fdf4a99f6b1f7e375d420c97b919d9fd67f6857bb9ea7ff6b47c34d6c32f WatchSource:0}: Error finding container 2cb5fdf4a99f6b1f7e375d420c97b919d9fd67f6857bb9ea7ff6b47c34d6c32f: Status 404 returned error can't find the container with id 2cb5fdf4a99f6b1f7e375d420c97b919d9fd67f6857bb9ea7ff6b47c34d6c32f
Sep 16 14:38:06.286080 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:06.286045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw" event={"ID":"65a81997-f339-4867-b807-ff104a37cc71","Type":"ContainerStarted","Data":"df1a85e8816e16d061b8b519b4dcd8c9039a3520fb25d88c415e4fc8e1681c49"}
Sep 16 14:38:06.286517 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:06.286155    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw" event={"ID":"65a81997-f339-4867-b807-ff104a37cc71","Type":"ContainerStarted","Data":"2cb5fdf4a99f6b1f7e375d420c97b919d9fd67f6857bb9ea7ff6b47c34d6c32f"}
Sep 16 14:38:06.286517 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:06.286190    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:06.287684 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:06.287661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq" event={"ID":"844a58b1-d795-4dc2-9673-546f4e4f9bf5","Type":"ContainerStarted","Data":"59c83b52fb7c21bbfd4f1b3f04ced6cdfa7b4dc193273a17b79d2d11b3ae46d3"}
Sep 16 14:38:06.287786 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:06.287701    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq" event={"ID":"844a58b1-d795-4dc2-9673-546f4e4f9bf5","Type":"ContainerStarted","Data":"17ab824d148e184fe49337f58b4b428b86221f0b9631812c79f477bfa4d03515"}
Sep 16 14:38:06.287786 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:06.287741    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:06.304232 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:06.304184    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw" podStartSLOduration=2.304169392 podStartE2EDuration="2.304169392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:38:06.303136936 +0000 UTC m=+1805.780369013" watchObservedRunningTime="2026-09-16 14:38:06.304169392 +0000 UTC m=+1805.781401467"
Sep 16 14:38:06.320317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:06.320276    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq" podStartSLOduration=2.320262382 podStartE2EDuration="2.320262382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:38:06.319649842 +0000 UTC m=+1805.796881939" watchObservedRunningTime="2026-09-16 14:38:06.320262382 +0000 UTC m=+1805.797494478"
Sep 16 14:38:12.396366 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:12.396331    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:38:17.295328 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:17.295291    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:17.295719 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:17.295480    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:17.972328 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:17.972291    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"]
Sep 16 14:38:17.975929 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:17.975909    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:17.978244 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:17.978223    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-5f0bee-dockercfg-v4zxp\""
Sep 16 14:38:17.978352 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:17.978224    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-5f0bee\""
Sep 16 14:38:17.988385 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:17.988364    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"]
Sep 16 14:38:18.011938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.011911    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"]
Sep 16 14:38:18.015121 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.015105    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:18.023994 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.023973    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"]
Sep 16 14:38:18.095335 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.095299    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdlkq\" (UniqueName: \"kubernetes.io/projected/5742cd69-36cc-4801-adee-fc1527246299-kube-api-access-gdlkq\") pod \"payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs\" (UID: \"5742cd69-36cc-4801-adee-fc1527246299\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:18.095509 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.095346    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5742cd69-36cc-4801-adee-fc1527246299-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs\" (UID: \"5742cd69-36cc-4801-adee-fc1527246299\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:18.095509 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.095367    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddkb8\" (UniqueName: \"kubernetes.io/projected/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-kube-api-access-ddkb8\") pod \"payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5\" (UID: \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:18.095509 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.095437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5\" (UID: \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:18.200613 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.197094    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdlkq\" (UniqueName: \"kubernetes.io/projected/5742cd69-36cc-4801-adee-fc1527246299-kube-api-access-gdlkq\") pod \"payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs\" (UID: \"5742cd69-36cc-4801-adee-fc1527246299\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:18.200613 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.197179    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5742cd69-36cc-4801-adee-fc1527246299-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs\" (UID: \"5742cd69-36cc-4801-adee-fc1527246299\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:18.200613 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.197218    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddkb8\" (UniqueName: \"kubernetes.io/projected/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-kube-api-access-ddkb8\") pod \"payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5\" (UID: \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:18.200613 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.197278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5\" (UID: \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:18.200613 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.198224    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5\" (UID: \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:18.200613 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.199368    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5742cd69-36cc-4801-adee-fc1527246299-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs\" (UID: \"5742cd69-36cc-4801-adee-fc1527246299\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:18.207754 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.207718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdlkq\" (UniqueName: \"kubernetes.io/projected/5742cd69-36cc-4801-adee-fc1527246299-kube-api-access-gdlkq\") pod \"payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs\" (UID: \"5742cd69-36cc-4801-adee-fc1527246299\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:18.208283 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.208265    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddkb8\" (UniqueName: \"kubernetes.io/projected/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-kube-api-access-ddkb8\") pod \"payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5\" (UID: \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:18.286278 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.286183    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:18.326467 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.326428    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:18.564707 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.564600    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"]
Sep 16 14:38:18.566958 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:38:18.566926    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f42b1bb_06f8_47a8_ab77_01aafd0ed67f.slice/crio-02c1be37dd975ad820fc55e1be8e4a37731cf0aeed9eb3a2db413d48be0d5065 WatchSource:0}: Error finding container 02c1be37dd975ad820fc55e1be8e4a37731cf0aeed9eb3a2db413d48be0d5065: Status 404 returned error can't find the container with id 02c1be37dd975ad820fc55e1be8e4a37731cf0aeed9eb3a2db413d48be0d5065
Sep 16 14:38:18.617207 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:18.617178    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"]
Sep 16 14:38:18.621197 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:38:18.621166    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5742cd69_36cc_4801_adee_fc1527246299.slice/crio-2c7584bb350038c9324fffdf811d62b6c631146f62efd676d6a88df9b3fc8b35 WatchSource:0}: Error finding container 2c7584bb350038c9324fffdf811d62b6c631146f62efd676d6a88df9b3fc8b35: Status 404 returned error can't find the container with id 2c7584bb350038c9324fffdf811d62b6c631146f62efd676d6a88df9b3fc8b35
Sep 16 14:38:19.355917 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:19.355882    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5" event={"ID":"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f","Type":"ContainerStarted","Data":"f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661"}
Sep 16 14:38:19.356402 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:19.355935    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5" event={"ID":"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f","Type":"ContainerStarted","Data":"02c1be37dd975ad820fc55e1be8e4a37731cf0aeed9eb3a2db413d48be0d5065"}
Sep 16 14:38:19.356402 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:19.356053    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:19.357620 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:19.357592    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs" event={"ID":"5742cd69-36cc-4801-adee-fc1527246299","Type":"ContainerStarted","Data":"6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6"}
Sep 16 14:38:19.357757 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:19.357638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs" event={"ID":"5742cd69-36cc-4801-adee-fc1527246299","Type":"ContainerStarted","Data":"2c7584bb350038c9324fffdf811d62b6c631146f62efd676d6a88df9b3fc8b35"}
Sep 16 14:38:19.357757 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:19.357665    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:19.378528 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:19.378454    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5" podStartSLOduration=2.378440495 podStartE2EDuration="2.378440495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:38:19.375452001 +0000 UTC m=+1818.852684076" watchObservedRunningTime="2026-09-16 14:38:19.378440495 +0000 UTC m=+1818.855672564"
Sep 16 14:38:19.397362 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:19.397319    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs" podStartSLOduration=2.397307205 podStartE2EDuration="2.397307205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:38:19.395341697 +0000 UTC m=+1818.872573816" watchObservedRunningTime="2026-09-16 14:38:19.397307205 +0000 UTC m=+1818.874539280"
Sep 16 14:38:30.364109 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:30.364072    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:38:30.364604 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:30.364152    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:38:32.516423 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:32.516383    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:38:37.085698 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:37.085667    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5_8f42b1bb-06f8-47a8-ab77-01aafd0ed67f/payload-processing/0.log"
Sep 16 14:38:41.420056 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:41.420025    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"]
Sep 16 14:38:41.423462 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:41.423446    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"
Sep 16 14:38:41.426041 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:41.426019    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:38:41.426190 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:41.426113    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:38:41.432659 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:41.432631    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"]
Sep 16 14:38:41.507962 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:41.507925    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042d3744-9d4f-4dd2-9ad4-e032200e3b32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a2fada-b22th\" (UID: \"042d3744-9d4f-4dd2-9ad4-e032200e3b32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"
Sep 16 14:38:41.608522 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:41.608480    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042d3744-9d4f-4dd2-9ad4-e032200e3b32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a2fada-b22th\" (UID: \"042d3744-9d4f-4dd2-9ad4-e032200e3b32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"
Sep 16 14:38:41.609113 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:41.609091    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042d3744-9d4f-4dd2-9ad4-e032200e3b32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a2fada-b22th\" (UID: \"042d3744-9d4f-4dd2-9ad4-e032200e3b32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"
Sep 16 14:38:41.736652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:41.736618    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"
Sep 16 14:38:42.006941 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:42.006856    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"]
Sep 16 14:38:42.010188 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:38:42.010161    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042d3744_9d4f_4dd2_9ad4_e032200e3b32.slice/crio-d57cff0013843dce4507658d53eb24e3dbeadb8cc17e13cda423237146786217 WatchSource:0}: Error finding container d57cff0013843dce4507658d53eb24e3dbeadb8cc17e13cda423237146786217: Status 404 returned error can't find the container with id d57cff0013843dce4507658d53eb24e3dbeadb8cc17e13cda423237146786217
Sep 16 14:38:42.471543 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:42.471500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th" event={"ID":"042d3744-9d4f-4dd2-9ad4-e032200e3b32","Type":"ContainerStarted","Data":"309651548c238138a283083f0d7bd7ae5eceaf47a667fc4f78de259000b24a28"}
Sep 16 14:38:42.471989 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:42.471559    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th" event={"ID":"042d3744-9d4f-4dd2-9ad4-e032200e3b32","Type":"ContainerStarted","Data":"d57cff0013843dce4507658d53eb24e3dbeadb8cc17e13cda423237146786217"}
Sep 16 14:38:42.489235 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:42.489191    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th" podStartSLOduration=1.489176741 podStartE2EDuration="1.489176741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:38:42.487757628 +0000 UTC m=+1841.964989706" watchObservedRunningTime="2026-09-16 14:38:42.489176741 +0000 UTC m=+1841.966408834"
Sep 16 14:38:43.477905 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:43.477863    2566 generic.go:356] "Generic (PLEG): container finished" podID="042d3744-9d4f-4dd2-9ad4-e032200e3b32" containerID="309651548c238138a283083f0d7bd7ae5eceaf47a667fc4f78de259000b24a28" exitCode=0
Sep 16 14:38:43.478300 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:43.477952    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th" event={"ID":"042d3744-9d4f-4dd2-9ad4-e032200e3b32","Type":"ContainerDied","Data":"309651548c238138a283083f0d7bd7ae5eceaf47a667fc4f78de259000b24a28"}
Sep 16 14:38:44.614285 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:44.614263    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"
Sep 16 14:38:44.737296 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:44.737210    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042d3744-9d4f-4dd2-9ad4-e032200e3b32-maas-api-service-ca\") pod \"042d3744-9d4f-4dd2-9ad4-e032200e3b32\" (UID: \"042d3744-9d4f-4dd2-9ad4-e032200e3b32\") "
Sep 16 14:38:44.737642 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:44.737621    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/042d3744-9d4f-4dd2-9ad4-e032200e3b32-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "042d3744-9d4f-4dd2-9ad4-e032200e3b32" (UID: "042d3744-9d4f-4dd2-9ad4-e032200e3b32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:38:44.838085 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:44.838049    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042d3744-9d4f-4dd2-9ad4-e032200e3b32-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:38:45.488654 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:45.488611    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th" event={"ID":"042d3744-9d4f-4dd2-9ad4-e032200e3b32","Type":"ContainerDied","Data":"d57cff0013843dce4507658d53eb24e3dbeadb8cc17e13cda423237146786217"}
Sep 16 14:38:45.488654 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:45.488642    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"
Sep 16 14:38:45.488862 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:45.488670    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d57cff0013843dce4507658d53eb24e3dbeadb8cc17e13cda423237146786217"
Sep 16 14:38:50.099930 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.099895    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"]
Sep 16 14:38:50.100276 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.100261    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="042d3744-9d4f-4dd2-9ad4-e032200e3b32" containerName="revoke-keys"
Sep 16 14:38:50.100276 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.100272    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="042d3744-9d4f-4dd2-9ad4-e032200e3b32" containerName="revoke-keys"
Sep 16 14:38:50.100361 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.100337    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="042d3744-9d4f-4dd2-9ad4-e032200e3b32" containerName="revoke-keys"
Sep 16 14:38:50.103413 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.103396    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.105923 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.105886    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-363d71\"/\"kube-root-ca.crt\""
Sep 16 14:38:50.106276 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.106165    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-363d71\"/\"openshift-service-ca.crt\""
Sep 16 14:38:50.106759 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.106732    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-363d71\"/\"ipp-route-363d71-kserve-self-signed-certs\""
Sep 16 14:38:50.106906 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.106748    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-363d71\"/\"default-dockercfg-cwqmz\""
Sep 16 14:38:50.115598 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.115579    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"]
Sep 16 14:38:50.286670 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.286632    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-tmp-dir\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.286842 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.286737    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7x4p\" (UniqueName: \"kubernetes.io/projected/3c02d947-b487-4eec-8a72-6300edfa5df0-kube-api-access-m7x4p\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.286842 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.286769    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-home\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.286842 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.286797    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-dshm\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.286954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.286841    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-model-cache\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.286954 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.286887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02d947-b487-4eec-8a72-6300edfa5df0-tls-certs\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.388237 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.388151    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02d947-b487-4eec-8a72-6300edfa5df0-tls-certs\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.388237 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.388196    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-tmp-dir\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.388452 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.388246    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7x4p\" (UniqueName: \"kubernetes.io/projected/3c02d947-b487-4eec-8a72-6300edfa5df0-kube-api-access-m7x4p\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.388452 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.388267    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-home\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.388452 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.388287    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-dshm\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.388452 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.388338    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-model-cache\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.388740 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.388721    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-model-cache\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.388803 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.388769    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-home\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.388803 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.388792    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-tmp-dir\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.390712 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.390689    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-dshm\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.390872 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.390856    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02d947-b487-4eec-8a72-6300edfa5df0-tls-certs\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.397212 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.397188    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7x4p\" (UniqueName: \"kubernetes.io/projected/3c02d947-b487-4eec-8a72-6300edfa5df0-kube-api-access-m7x4p\") pod \"ipp-route-363d71-kserve-6b6966dbdc-2nj6j\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") " pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.413744 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.413718    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:38:50.706703 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:50.706672    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"]
Sep 16 14:38:50.708348 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:38:50.708320    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c02d947_b487_4eec_8a72_6300edfa5df0.slice/crio-50feed5b955e6a386485a71926fb78d3a7045a4d7f32724a172f0b3ef8eda8d8 WatchSource:0}: Error finding container 50feed5b955e6a386485a71926fb78d3a7045a4d7f32724a172f0b3ef8eda8d8: Status 404 returned error can't find the container with id 50feed5b955e6a386485a71926fb78d3a7045a4d7f32724a172f0b3ef8eda8d8
Sep 16 14:38:51.518947 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:51.518912    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j" event={"ID":"3c02d947-b487-4eec-8a72-6300edfa5df0","Type":"ContainerStarted","Data":"2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d"}
Sep 16 14:38:51.518947 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:51.518953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j" event={"ID":"3c02d947-b487-4eec-8a72-6300edfa5df0","Type":"ContainerStarted","Data":"50feed5b955e6a386485a71926fb78d3a7045a4d7f32724a172f0b3ef8eda8d8"}
Sep 16 14:38:51.544824 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:51.544770    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j" podStartSLOduration=1.544754632 podStartE2EDuration="1.544754632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:38:51.543359777 +0000 UTC m=+1851.020591851" watchObservedRunningTime="2026-09-16 14:38:51.544754632 +0000 UTC m=+1851.021986706"
Sep 16 14:38:57.369730 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.369689    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"]
Sep 16 14:38:57.370208 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.369900    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw" podUID="65a81997-f339-4867-b807-ff104a37cc71" containerName="payload-processing" containerID="cri-o://df1a85e8816e16d061b8b519b4dcd8c9039a3520fb25d88c415e4fc8e1681c49" gracePeriod=30
Sep 16 14:38:57.377008 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.376984    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"]
Sep 16 14:38:57.377199 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.377177    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq" podUID="844a58b1-d795-4dc2-9673-546f4e4f9bf5" containerName="payload-pre-processing" containerID="cri-o://59c83b52fb7c21bbfd4f1b3f04ced6cdfa7b4dc193273a17b79d2d11b3ae46d3" gracePeriod=30
Sep 16 14:38:57.500232 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.500201    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:38:57.555589 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.555395    2566 generic.go:356] "Generic (PLEG): container finished" podID="65a81997-f339-4867-b807-ff104a37cc71" containerID="df1a85e8816e16d061b8b519b4dcd8c9039a3520fb25d88c415e4fc8e1681c49" exitCode=0
Sep 16 14:38:57.555748 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.555632    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw" event={"ID":"65a81997-f339-4867-b807-ff104a37cc71","Type":"ContainerDied","Data":"df1a85e8816e16d061b8b519b4dcd8c9039a3520fb25d88c415e4fc8e1681c49"}
Sep 16 14:38:57.557955 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.557765    2566 generic.go:356] "Generic (PLEG): container finished" podID="844a58b1-d795-4dc2-9673-546f4e4f9bf5" containerID="59c83b52fb7c21bbfd4f1b3f04ced6cdfa7b4dc193273a17b79d2d11b3ae46d3" exitCode=0
Sep 16 14:38:57.557955 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.557844    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq" event={"ID":"844a58b1-d795-4dc2-9673-546f4e4f9bf5","Type":"ContainerDied","Data":"59c83b52fb7c21bbfd4f1b3f04ced6cdfa7b4dc193273a17b79d2d11b3ae46d3"}
Sep 16 14:38:57.599489 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.599457    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"]
Sep 16 14:38:57.602440 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.602405    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a2fada-b22th"]
Sep 16 14:38:57.652374 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.652352    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:57.655648 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.655619    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:57.746650 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.746615    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a81997-f339-4867-b807-ff104a37cc71-plugins-config-volume\") pod \"65a81997-f339-4867-b807-ff104a37cc71\" (UID: \"65a81997-f339-4867-b807-ff104a37cc71\") "
Sep 16 14:38:57.746818 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.746725    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a58b1-d795-4dc2-9673-546f4e4f9bf5-plugins-config-volume\") pod \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\" (UID: \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\") "
Sep 16 14:38:57.746818 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.746759    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65bbm\" (UniqueName: \"kubernetes.io/projected/65a81997-f339-4867-b807-ff104a37cc71-kube-api-access-65bbm\") pod \"65a81997-f339-4867-b807-ff104a37cc71\" (UID: \"65a81997-f339-4867-b807-ff104a37cc71\") "
Sep 16 14:38:57.746818 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.746810    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h84hq\" (UniqueName: \"kubernetes.io/projected/844a58b1-d795-4dc2-9673-546f4e4f9bf5-kube-api-access-h84hq\") pod \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\" (UID: \"844a58b1-d795-4dc2-9673-546f4e4f9bf5\") "
Sep 16 14:38:57.747066 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.747042    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65a81997-f339-4867-b807-ff104a37cc71-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65a81997-f339-4867-b807-ff104a37cc71" (UID: "65a81997-f339-4867-b807-ff104a37cc71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:38:57.747145 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.747092    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/844a58b1-d795-4dc2-9673-546f4e4f9bf5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "844a58b1-d795-4dc2-9673-546f4e4f9bf5" (UID: "844a58b1-d795-4dc2-9673-546f4e4f9bf5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:38:57.749066 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.749038    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/844a58b1-d795-4dc2-9673-546f4e4f9bf5-kube-api-access-h84hq" (OuterVolumeSpecName: "kube-api-access-h84hq") pod "844a58b1-d795-4dc2-9673-546f4e4f9bf5" (UID: "844a58b1-d795-4dc2-9673-546f4e4f9bf5"). InnerVolumeSpecName "kube-api-access-h84hq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:38:57.749066 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.749054    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65a81997-f339-4867-b807-ff104a37cc71-kube-api-access-65bbm" (OuterVolumeSpecName: "kube-api-access-65bbm") pod "65a81997-f339-4867-b807-ff104a37cc71" (UID: "65a81997-f339-4867-b807-ff104a37cc71"). InnerVolumeSpecName "kube-api-access-65bbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:38:57.847892 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.847850    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a81997-f339-4867-b807-ff104a37cc71-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:38:57.847892 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.847884    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844a58b1-d795-4dc2-9673-546f4e4f9bf5-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:38:57.847892 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.847897    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65bbm\" (UniqueName: \"kubernetes.io/projected/65a81997-f339-4867-b807-ff104a37cc71-kube-api-access-65bbm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:38:57.848132 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:57.847910    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h84hq\" (UniqueName: \"kubernetes.io/projected/844a58b1-d795-4dc2-9673-546f4e4f9bf5-kube-api-access-h84hq\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:38:58.563356 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.563323    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"
Sep 16 14:38:58.563828 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.563365    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq" event={"ID":"844a58b1-d795-4dc2-9673-546f4e4f9bf5","Type":"ContainerDied","Data":"17ab824d148e184fe49337f58b4b428b86221f0b9631812c79f477bfa4d03515"}
Sep 16 14:38:58.563828 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.563434    2566 scope.go:117] "RemoveContainer" containerID="59c83b52fb7c21bbfd4f1b3f04ced6cdfa7b4dc193273a17b79d2d11b3ae46d3"
Sep 16 14:38:58.564835 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.564816    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw" event={"ID":"65a81997-f339-4867-b807-ff104a37cc71","Type":"ContainerDied","Data":"2cb5fdf4a99f6b1f7e375d420c97b919d9fd67f6857bb9ea7ff6b47c34d6c32f"}
Sep 16 14:38:58.564905 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.564849    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"
Sep 16 14:38:58.573458 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.573441    2566 scope.go:117] "RemoveContainer" containerID="df1a85e8816e16d061b8b519b4dcd8c9039a3520fb25d88c415e4fc8e1681c49"
Sep 16 14:38:58.588415 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.588396    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"]
Sep 16 14:38:58.597381 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.597363    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2fada-855bc58b6-cbgnq"]
Sep 16 14:38:58.610430 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.610407    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"]
Sep 16 14:38:58.617213 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.613867    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2fada-7c4499b6fc-v7mmw"]
Sep 16 14:38:58.984874 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.984844    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="042d3744-9d4f-4dd2-9ad4-e032200e3b32" path="/var/lib/kubelet/pods/042d3744-9d4f-4dd2-9ad4-e032200e3b32/volumes"
Sep 16 14:38:58.985204 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.985191    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65a81997-f339-4867-b807-ff104a37cc71" path="/var/lib/kubelet/pods/65a81997-f339-4867-b807-ff104a37cc71/volumes"
Sep 16 14:38:58.985498 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:38:58.985487    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="844a58b1-d795-4dc2-9673-546f4e4f9bf5" path="/var/lib/kubelet/pods/844a58b1-d795-4dc2-9673-546f4e4f9bf5/volumes"
Sep 16 14:39:00.414745 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:00.414707    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:39:00.415172 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:00.414765    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:39:00.428009 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:00.427980    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:39:00.588129 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:00.588096    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:39:12.794853 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:12.794819    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:39:19.874161 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.874125    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"]
Sep 16 14:39:19.874754 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.874686    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="844a58b1-d795-4dc2-9673-546f4e4f9bf5" containerName="payload-pre-processing"
Sep 16 14:39:19.874754 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.874706    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="844a58b1-d795-4dc2-9673-546f4e4f9bf5" containerName="payload-pre-processing"
Sep 16 14:39:19.874754 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.874743    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65a81997-f339-4867-b807-ff104a37cc71" containerName="payload-processing"
Sep 16 14:39:19.874754 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.874752    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="65a81997-f339-4867-b807-ff104a37cc71" containerName="payload-processing"
Sep 16 14:39:19.874983 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.874828    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="65a81997-f339-4867-b807-ff104a37cc71" containerName="payload-processing"
Sep 16 14:39:19.874983 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.874841    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="844a58b1-d795-4dc2-9673-546f4e4f9bf5" containerName="payload-pre-processing"
Sep 16 14:39:19.878150 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.878131    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"
Sep 16 14:39:19.880656 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.880625    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:39:19.880767 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.880671    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:39:19.884929 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.884907    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"]
Sep 16 14:39:19.937622 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:19.937577    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716b1375-1f0e-4065-8080-f94532693de6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-576150-t5bxd\" (UID: \"716b1375-1f0e-4065-8080-f94532693de6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"
Sep 16 14:39:20.039079 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:20.039029    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716b1375-1f0e-4065-8080-f94532693de6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-576150-t5bxd\" (UID: \"716b1375-1f0e-4065-8080-f94532693de6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"
Sep 16 14:39:20.039709 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:20.039688    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716b1375-1f0e-4065-8080-f94532693de6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-576150-t5bxd\" (UID: \"716b1375-1f0e-4065-8080-f94532693de6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"
Sep 16 14:39:20.189281 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:20.189193    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"
Sep 16 14:39:20.465304 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:20.465278    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"]
Sep 16 14:39:20.467044 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:39:20.467015    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod716b1375_1f0e_4065_8080_f94532693de6.slice/crio-2b6471535d72ea7fcea58d5069e57ab0672ea355927ed8203284cd1d2ef46ccf WatchSource:0}: Error finding container 2b6471535d72ea7fcea58d5069e57ab0672ea355927ed8203284cd1d2ef46ccf: Status 404 returned error can't find the container with id 2b6471535d72ea7fcea58d5069e57ab0672ea355927ed8203284cd1d2ef46ccf
Sep 16 14:39:20.672215 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:20.672180    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd" event={"ID":"716b1375-1f0e-4065-8080-f94532693de6","Type":"ContainerStarted","Data":"2b6471535d72ea7fcea58d5069e57ab0672ea355927ed8203284cd1d2ef46ccf"}
Sep 16 14:39:21.679406 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:21.679375    2566 generic.go:356] "Generic (PLEG): container finished" podID="716b1375-1f0e-4065-8080-f94532693de6" containerID="b411395cdb88dfd20d13ce5af2afef7a8d05468625e5fe52fd16ebcb7e972360" exitCode=0
Sep 16 14:39:21.679942 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:21.679461    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd" event={"ID":"716b1375-1f0e-4065-8080-f94532693de6","Type":"ContainerDied","Data":"b411395cdb88dfd20d13ce5af2afef7a8d05468625e5fe52fd16ebcb7e972360"}
Sep 16 14:39:22.817890 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:22.817864    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"
Sep 16 14:39:22.862696 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:22.862670    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716b1375-1f0e-4065-8080-f94532693de6-maas-api-service-ca\") pod \"716b1375-1f0e-4065-8080-f94532693de6\" (UID: \"716b1375-1f0e-4065-8080-f94532693de6\") "
Sep 16 14:39:22.863075 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:22.863049    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/716b1375-1f0e-4065-8080-f94532693de6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "716b1375-1f0e-4065-8080-f94532693de6" (UID: "716b1375-1f0e-4065-8080-f94532693de6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:39:22.964093 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:22.964057    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716b1375-1f0e-4065-8080-f94532693de6-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:39:23.691418 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:23.691381    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd" event={"ID":"716b1375-1f0e-4065-8080-f94532693de6","Type":"ContainerDied","Data":"2b6471535d72ea7fcea58d5069e57ab0672ea355927ed8203284cd1d2ef46ccf"}
Sep 16 14:39:23.691418 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:23.691421    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"
Sep 16 14:39:23.691651 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:23.691441    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b6471535d72ea7fcea58d5069e57ab0672ea355927ed8203284cd1d2ef46ccf"
Sep 16 14:39:28.504658 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:28.504622    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:39:35.359886 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:35.359855    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht_946bc9b8-228d-4203-835e-634c615c7a03/payload-processing/0.log"
Sep 16 14:39:35.702729 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:35.702701    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"]
Sep 16 14:39:35.706407 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:35.706387    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-576150-t5bxd"]
Sep 16 14:39:35.730021 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:35.729995    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5_8f42b1bb-06f8-47a8-ab77-01aafd0ed67f/payload-processing/0.log"
Sep 16 14:39:35.861181 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:35.861138    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5_8f42b1bb-06f8-47a8-ab77-01aafd0ed67f/payload-processing/0.log"
Sep 16 14:39:36.983956 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:36.983921    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="716b1375-1f0e-4065-8080-f94532693de6" path="/var/lib/kubelet/pods/716b1375-1f0e-4065-8080-f94532693de6/volumes"
Sep 16 14:39:42.541719 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.541689    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"]
Sep 16 14:39:42.542094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.542053    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="716b1375-1f0e-4065-8080-f94532693de6" containerName="revoke-keys"
Sep 16 14:39:42.542094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.542064    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="716b1375-1f0e-4065-8080-f94532693de6" containerName="revoke-keys"
Sep 16 14:39:42.542170 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.542132    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="716b1375-1f0e-4065-8080-f94532693de6" containerName="revoke-keys"
Sep 16 14:39:42.545151 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.545134    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:42.548144 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.548122    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-d5498f-dockercfg-htxtl\""
Sep 16 14:39:42.549017 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.548995    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d5498f\""
Sep 16 14:39:42.556285 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.556263    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"]
Sep 16 14:39:42.640270 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.640236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpfsd\" (UniqueName: \"kubernetes.io/projected/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-kube-api-access-dpfsd\") pod \"payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp\" (UID: \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:42.640431 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.640312    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp\" (UID: \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:42.741103 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.741065    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfsd\" (UniqueName: \"kubernetes.io/projected/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-kube-api-access-dpfsd\") pod \"payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp\" (UID: \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:42.741274 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.741116    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp\" (UID: \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:42.741840 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.741822    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp\" (UID: \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:42.749223 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.749197    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpfsd\" (UniqueName: \"kubernetes.io/projected/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-kube-api-access-dpfsd\") pod \"payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp\" (UID: \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:42.855447 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:42.855356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:43.141853 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:43.141829    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"]
Sep 16 14:39:43.143271 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:39:43.143243    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb15b4d8_1ca3_4100_bd0b_298dd4948b51.slice/crio-63bc98745e86d67e7af1792a6d8c7e028856cd5573dbd4197071eb693d5a75be WatchSource:0}: Error finding container 63bc98745e86d67e7af1792a6d8c7e028856cd5573dbd4197071eb693d5a75be: Status 404 returned error can't find the container with id 63bc98745e86d67e7af1792a6d8c7e028856cd5573dbd4197071eb693d5a75be
Sep 16 14:39:43.788229 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:43.788196    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp" event={"ID":"bb15b4d8-1ca3-4100-bd0b-298dd4948b51","Type":"ContainerStarted","Data":"75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6"}
Sep 16 14:39:43.788663 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:43.788253    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp" event={"ID":"bb15b4d8-1ca3-4100-bd0b-298dd4948b51","Type":"ContainerStarted","Data":"63bc98745e86d67e7af1792a6d8c7e028856cd5573dbd4197071eb693d5a75be"}
Sep 16 14:39:43.788663 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:43.788322    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:43.813011 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:43.812962    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp" podStartSLOduration=1.8129446809999998 podStartE2EDuration="1.812944681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:39:43.810558631 +0000 UTC m=+1903.287790738" watchObservedRunningTime="2026-09-16 14:39:43.812944681 +0000 UTC m=+1903.290176799"
Sep 16 14:39:45.000837 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.000802    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"]
Sep 16 14:39:45.004075 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.004057    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"
Sep 16 14:39:45.008387 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.008364    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:39:45.008740 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.008723    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:39:45.019324 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.019296    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"]
Sep 16 14:39:45.164328 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.164286    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6cb1dc-e2ee-405b-ab09-17c6a89541e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6\" (UID: \"6e6cb1dc-e2ee-405b-ab09-17c6a89541e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"
Sep 16 14:39:45.264848 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.264774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6cb1dc-e2ee-405b-ab09-17c6a89541e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6\" (UID: \"6e6cb1dc-e2ee-405b-ab09-17c6a89541e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"
Sep 16 14:39:45.265362 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.265343    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6cb1dc-e2ee-405b-ab09-17c6a89541e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6\" (UID: \"6e6cb1dc-e2ee-405b-ab09-17c6a89541e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"
Sep 16 14:39:45.314033 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.313999    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"
Sep 16 14:39:45.585788 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.585759    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"]
Sep 16 14:39:45.587671 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:39:45.587633    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e6cb1dc_e2ee_405b_ab09_17c6a89541e4.slice/crio-ffd110fb7266a25d1a01f6b165584b9cf3256bab051a8b3d954d9e6a75c6f5bb WatchSource:0}: Error finding container ffd110fb7266a25d1a01f6b165584b9cf3256bab051a8b3d954d9e6a75c6f5bb: Status 404 returned error can't find the container with id ffd110fb7266a25d1a01f6b165584b9cf3256bab051a8b3d954d9e6a75c6f5bb
Sep 16 14:39:45.799788 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:45.799754    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6" event={"ID":"6e6cb1dc-e2ee-405b-ab09-17c6a89541e4","Type":"ContainerStarted","Data":"ffd110fb7266a25d1a01f6b165584b9cf3256bab051a8b3d954d9e6a75c6f5bb"}
Sep 16 14:39:46.806203 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:46.806170    2566 generic.go:356] "Generic (PLEG): container finished" podID="6e6cb1dc-e2ee-405b-ab09-17c6a89541e4" containerID="bf5c9e5e2449336ca7bd19ff19020e87297b59802c0f451d177ab762604ce4f9" exitCode=0
Sep 16 14:39:46.806603 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:46.806252    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6" event={"ID":"6e6cb1dc-e2ee-405b-ab09-17c6a89541e4","Type":"ContainerDied","Data":"bf5c9e5e2449336ca7bd19ff19020e87297b59802c0f451d177ab762604ce4f9"}
Sep 16 14:39:47.947353 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:47.947323    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"
Sep 16 14:39:48.093378 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:48.093292    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6cb1dc-e2ee-405b-ab09-17c6a89541e4-maas-api-service-ca\") pod \"6e6cb1dc-e2ee-405b-ab09-17c6a89541e4\" (UID: \"6e6cb1dc-e2ee-405b-ab09-17c6a89541e4\") "
Sep 16 14:39:48.093665 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:48.093644    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e6cb1dc-e2ee-405b-ab09-17c6a89541e4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e6cb1dc-e2ee-405b-ab09-17c6a89541e4" (UID: "6e6cb1dc-e2ee-405b-ab09-17c6a89541e4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:39:48.194948 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:48.194914    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e6cb1dc-e2ee-405b-ab09-17c6a89541e4-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:39:48.817702 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:48.817669    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"
Sep 16 14:39:48.817884 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:48.817720    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6" event={"ID":"6e6cb1dc-e2ee-405b-ab09-17c6a89541e4","Type":"ContainerDied","Data":"ffd110fb7266a25d1a01f6b165584b9cf3256bab051a8b3d954d9e6a75c6f5bb"}
Sep 16 14:39:48.817884 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:48.817771    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffd110fb7266a25d1a01f6b165584b9cf3256bab051a8b3d954d9e6a75c6f5bb"
Sep 16 14:39:52.326629 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:52.326592    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:39:54.795694 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:54.795665    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:39:56.100896 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:56.100858    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"]
Sep 16 14:39:56.105597 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:56.105533    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-dc3e5b-cr7s6"]
Sep 16 14:39:56.984444 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:39:56.984408    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e6cb1dc-e2ee-405b-ab09-17c6a89541e4" path="/var/lib/kubelet/pods/6e6cb1dc-e2ee-405b-ab09-17c6a89541e4/volumes"
Sep 16 14:40:05.588681 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.588642    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"]
Sep 16 14:40:05.589279 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.589261    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e6cb1dc-e2ee-405b-ab09-17c6a89541e4" containerName="revoke-keys"
Sep 16 14:40:05.589359 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.589283    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e6cb1dc-e2ee-405b-ab09-17c6a89541e4" containerName="revoke-keys"
Sep 16 14:40:05.589402 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.589386    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e6cb1dc-e2ee-405b-ab09-17c6a89541e4" containerName="revoke-keys"
Sep 16 14:40:05.592737 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.592720    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"
Sep 16 14:40:05.595178 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.595141    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:40:05.596106 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.595249    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:40:05.598951 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.598924    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"]
Sep 16 14:40:05.659504 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.659472    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3698ec3-8f15-4b6c-8fba-7139a502a4c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb\" (UID: \"a3698ec3-8f15-4b6c-8fba-7139a502a4c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"
Sep 16 14:40:05.759961 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.759920    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3698ec3-8f15-4b6c-8fba-7139a502a4c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb\" (UID: \"a3698ec3-8f15-4b6c-8fba-7139a502a4c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"
Sep 16 14:40:05.760581 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.760537    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3698ec3-8f15-4b6c-8fba-7139a502a4c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb\" (UID: \"a3698ec3-8f15-4b6c-8fba-7139a502a4c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"
Sep 16 14:40:05.906466 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:05.906383    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"
Sep 16 14:40:06.182888 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:06.182860    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"]
Sep 16 14:40:06.185134 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:40:06.185099    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3698ec3_8f15_4b6c_8fba_7139a502a4c7.slice/crio-914203ba5a076398eb215d172b306b9cfe1be49f60340d248420c21eb11aae9a WatchSource:0}: Error finding container 914203ba5a076398eb215d172b306b9cfe1be49f60340d248420c21eb11aae9a: Status 404 returned error can't find the container with id 914203ba5a076398eb215d172b306b9cfe1be49f60340d248420c21eb11aae9a
Sep 16 14:40:06.910592 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:06.910542    2566 generic.go:356] "Generic (PLEG): container finished" podID="a3698ec3-8f15-4b6c-8fba-7139a502a4c7" containerID="2f09e2861075a63d44df8b85722bdbcef29cf65c371fdca8cc64c25ad0db18fe" exitCode=0
Sep 16 14:40:06.910931 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:06.910633    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb" event={"ID":"a3698ec3-8f15-4b6c-8fba-7139a502a4c7","Type":"ContainerDied","Data":"2f09e2861075a63d44df8b85722bdbcef29cf65c371fdca8cc64c25ad0db18fe"}
Sep 16 14:40:06.910931 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:06.910690    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb" event={"ID":"a3698ec3-8f15-4b6c-8fba-7139a502a4c7","Type":"ContainerStarted","Data":"914203ba5a076398eb215d172b306b9cfe1be49f60340d248420c21eb11aae9a"}
Sep 16 14:40:08.053709 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:08.053685    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"
Sep 16 14:40:08.182800 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:08.182717    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3698ec3-8f15-4b6c-8fba-7139a502a4c7-maas-api-service-ca\") pod \"a3698ec3-8f15-4b6c-8fba-7139a502a4c7\" (UID: \"a3698ec3-8f15-4b6c-8fba-7139a502a4c7\") "
Sep 16 14:40:08.183067 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:08.183042    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3698ec3-8f15-4b6c-8fba-7139a502a4c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a3698ec3-8f15-4b6c-8fba-7139a502a4c7" (UID: "a3698ec3-8f15-4b6c-8fba-7139a502a4c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:08.284145 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:08.284091    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3698ec3-8f15-4b6c-8fba-7139a502a4c7-maas-api-service-ca\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:40:08.922209 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:08.922177    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"
Sep 16 14:40:08.922395 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:08.922217    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb" event={"ID":"a3698ec3-8f15-4b6c-8fba-7139a502a4c7","Type":"ContainerDied","Data":"914203ba5a076398eb215d172b306b9cfe1be49f60340d248420c21eb11aae9a"}
Sep 16 14:40:08.922395 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:08.922270    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="914203ba5a076398eb215d172b306b9cfe1be49f60340d248420c21eb11aae9a"
Sep 16 14:40:10.197918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:10.197873    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:40:12.277315 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.277286    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"]
Sep 16 14:40:12.277761 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.277477    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp" podUID="bb15b4d8-1ca3-4100-bd0b-298dd4948b51" containerName="payload-processing" containerID="cri-o://75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6" gracePeriod=30
Sep 16 14:40:12.525428 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.525400    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:40:12.542552 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.542483    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"]
Sep 16 14:40:12.553379 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.553346    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5498f-5bldb"]
Sep 16 14:40:12.620000 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.619970    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpfsd\" (UniqueName: \"kubernetes.io/projected/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-kube-api-access-dpfsd\") pod \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\" (UID: \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\") "
Sep 16 14:40:12.620174 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.620049    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-plugins-config-volume\") pod \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\" (UID: \"bb15b4d8-1ca3-4100-bd0b-298dd4948b51\") "
Sep 16 14:40:12.620419 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.620394    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb15b4d8-1ca3-4100-bd0b-298dd4948b51" (UID: "bb15b4d8-1ca3-4100-bd0b-298dd4948b51"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:12.622255 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.622228    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-kube-api-access-dpfsd" (OuterVolumeSpecName: "kube-api-access-dpfsd") pod "bb15b4d8-1ca3-4100-bd0b-298dd4948b51" (UID: "bb15b4d8-1ca3-4100-bd0b-298dd4948b51"). InnerVolumeSpecName "kube-api-access-dpfsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:12.721410 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.721376    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:40:12.721410 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.721406    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpfsd\" (UniqueName: \"kubernetes.io/projected/bb15b4d8-1ca3-4100-bd0b-298dd4948b51-kube-api-access-dpfsd\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:40:12.943154 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.943120    2566 generic.go:356] "Generic (PLEG): container finished" podID="bb15b4d8-1ca3-4100-bd0b-298dd4948b51" containerID="75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6" exitCode=0
Sep 16 14:40:12.943316 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.943164    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"
Sep 16 14:40:12.943316 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.943210    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp" event={"ID":"bb15b4d8-1ca3-4100-bd0b-298dd4948b51","Type":"ContainerDied","Data":"75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6"}
Sep 16 14:40:12.943316 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.943266    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp" event={"ID":"bb15b4d8-1ca3-4100-bd0b-298dd4948b51","Type":"ContainerDied","Data":"63bc98745e86d67e7af1792a6d8c7e028856cd5573dbd4197071eb693d5a75be"}
Sep 16 14:40:12.943316 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.943297    2566 scope.go:117] "RemoveContainer" containerID="75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6"
Sep 16 14:40:12.952589 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.952553    2566 scope.go:117] "RemoveContainer" containerID="75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6"
Sep 16 14:40:12.952861 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:40:12.952837    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6\": container with ID starting with 75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6 not found: ID does not exist" containerID="75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6"
Sep 16 14:40:12.952924 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.952870    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6"} err="failed to get container status \"75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6\": rpc error: code = NotFound desc = could not find container \"75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6\": container with ID starting with 75d9f8fed08ca7a1ae7bf51c15d664d94bf1ffe0a4285fb12da8417f2dcf0cd6 not found: ID does not exist"
Sep 16 14:40:12.967721 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.967699    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"]
Sep 16 14:40:12.970351 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.970329    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5498f-d694d7cd7-lqckp"]
Sep 16 14:40:12.983486 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.983459    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3698ec3-8f15-4b6c-8fba-7139a502a4c7" path="/var/lib/kubelet/pods/a3698ec3-8f15-4b6c-8fba-7139a502a4c7/volumes"
Sep 16 14:40:12.983896 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:12.983875    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb15b4d8-1ca3-4100-bd0b-298dd4948b51" path="/var/lib/kubelet/pods/bb15b4d8-1ca3-4100-bd0b-298dd4948b51/volumes"
Sep 16 14:40:24.916253 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:24.916214    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:40:40.191671 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:40.191635    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:40:40.861534 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:40.861501    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"]
Sep 16 14:40:40.861812 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:40.861768    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht" podUID="946bc9b8-228d-4203-835e-634c615c7a03" containerName="payload-processing" containerID="cri-o://ba20ff29447b845372683deef1ea550097da148a7d6a915b2c948b03d51f98d0" gracePeriod=30
Sep 16 14:40:41.082626 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:41.082555    2566 generic.go:356] "Generic (PLEG): container finished" podID="946bc9b8-228d-4203-835e-634c615c7a03" containerID="ba20ff29447b845372683deef1ea550097da148a7d6a915b2c948b03d51f98d0" exitCode=0
Sep 16 14:40:41.082792 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:41.082633    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht" event={"ID":"946bc9b8-228d-4203-835e-634c615c7a03","Type":"ContainerDied","Data":"ba20ff29447b845372683deef1ea550097da148a7d6a915b2c948b03d51f98d0"}
Sep 16 14:40:41.109934 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:41.109907    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:40:41.168293 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:41.168209    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74n69\" (UniqueName: \"kubernetes.io/projected/946bc9b8-228d-4203-835e-634c615c7a03-kube-api-access-74n69\") pod \"946bc9b8-228d-4203-835e-634c615c7a03\" (UID: \"946bc9b8-228d-4203-835e-634c615c7a03\") "
Sep 16 14:40:41.168467 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:41.168298    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/946bc9b8-228d-4203-835e-634c615c7a03-plugins-config-volume\") pod \"946bc9b8-228d-4203-835e-634c615c7a03\" (UID: \"946bc9b8-228d-4203-835e-634c615c7a03\") "
Sep 16 14:40:41.168792 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:41.168761    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/946bc9b8-228d-4203-835e-634c615c7a03-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "946bc9b8-228d-4203-835e-634c615c7a03" (UID: "946bc9b8-228d-4203-835e-634c615c7a03"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:41.170600 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:41.170577    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946bc9b8-228d-4203-835e-634c615c7a03-kube-api-access-74n69" (OuterVolumeSpecName: "kube-api-access-74n69") pod "946bc9b8-228d-4203-835e-634c615c7a03" (UID: "946bc9b8-228d-4203-835e-634c615c7a03"). InnerVolumeSpecName "kube-api-access-74n69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:41.269216 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:41.269168    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74n69\" (UniqueName: \"kubernetes.io/projected/946bc9b8-228d-4203-835e-634c615c7a03-kube-api-access-74n69\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:40:41.269216 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:41.269202    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/946bc9b8-228d-4203-835e-634c615c7a03-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:40:42.090022 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:42.089992    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"
Sep 16 14:40:42.090022 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:42.090013    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht" event={"ID":"946bc9b8-228d-4203-835e-634c615c7a03","Type":"ContainerDied","Data":"ecfababa05c97ce0c7ce1ed47ffd919187f5cf181f7370a61f88020f58ece43b"}
Sep 16 14:40:42.090240 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:42.090070    2566 scope.go:117] "RemoveContainer" containerID="ba20ff29447b845372683deef1ea550097da148a7d6a915b2c948b03d51f98d0"
Sep 16 14:40:42.123093 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:42.123063    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"]
Sep 16 14:40:42.128400 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:42.128376    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-363d71-755fcd8649-h7fht"]
Sep 16 14:40:42.984208 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:42.984177    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946bc9b8-228d-4203-835e-634c615c7a03" path="/var/lib/kubelet/pods/946bc9b8-228d-4203-835e-634c615c7a03/volumes"
Sep 16 14:40:46.692632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:46.692589    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"]
Sep 16 14:40:49.996086 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:40:49.996055    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:41:01.712360 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:01.712313    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j" podUID="3c02d947-b487-4eec-8a72-6300edfa5df0" containerName="main" containerID="cri-o://2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d" gracePeriod=45
Sep 16 14:41:01.967491 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:01.967423    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:41:02.055129 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055100    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-model-cache\") pod \"3c02d947-b487-4eec-8a72-6300edfa5df0\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") "
Sep 16 14:41:02.055305 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055149    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-dshm\") pod \"3c02d947-b487-4eec-8a72-6300edfa5df0\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") "
Sep 16 14:41:02.055305 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055211    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7x4p\" (UniqueName: \"kubernetes.io/projected/3c02d947-b487-4eec-8a72-6300edfa5df0-kube-api-access-m7x4p\") pod \"3c02d947-b487-4eec-8a72-6300edfa5df0\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") "
Sep 16 14:41:02.055305 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055234    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02d947-b487-4eec-8a72-6300edfa5df0-tls-certs\") pod \"3c02d947-b487-4eec-8a72-6300edfa5df0\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") "
Sep 16 14:41:02.055305 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055266    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-tmp-dir\") pod \"3c02d947-b487-4eec-8a72-6300edfa5df0\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") "
Sep 16 14:41:02.055535 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055314    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-home\") pod \"3c02d947-b487-4eec-8a72-6300edfa5df0\" (UID: \"3c02d947-b487-4eec-8a72-6300edfa5df0\") "
Sep 16 14:41:02.055535 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055413    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c02d947-b487-4eec-8a72-6300edfa5df0" (UID: "3c02d947-b487-4eec-8a72-6300edfa5df0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:41:02.055662 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055627    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c02d947-b487-4eec-8a72-6300edfa5df0" (UID: "3c02d947-b487-4eec-8a72-6300edfa5df0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:41:02.055741 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055716    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:02.055801 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.055741    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:02.056054 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.056029    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-home" (OuterVolumeSpecName: "home") pod "3c02d947-b487-4eec-8a72-6300edfa5df0" (UID: "3c02d947-b487-4eec-8a72-6300edfa5df0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:41:02.057774 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.057749    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c02d947-b487-4eec-8a72-6300edfa5df0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c02d947-b487-4eec-8a72-6300edfa5df0" (UID: "3c02d947-b487-4eec-8a72-6300edfa5df0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:41:02.057774 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.057760    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c02d947-b487-4eec-8a72-6300edfa5df0-kube-api-access-m7x4p" (OuterVolumeSpecName: "kube-api-access-m7x4p") pod "3c02d947-b487-4eec-8a72-6300edfa5df0" (UID: "3c02d947-b487-4eec-8a72-6300edfa5df0"). InnerVolumeSpecName "kube-api-access-m7x4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:41:02.057918 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.057753    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-dshm" (OuterVolumeSpecName: "dshm") pod "3c02d947-b487-4eec-8a72-6300edfa5df0" (UID: "3c02d947-b487-4eec-8a72-6300edfa5df0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:41:02.157094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.157063    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:02.157094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.157092    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c02d947-b487-4eec-8a72-6300edfa5df0-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:02.157298 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.157105    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7x4p\" (UniqueName: \"kubernetes.io/projected/3c02d947-b487-4eec-8a72-6300edfa5df0-kube-api-access-m7x4p\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:02.157298 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.157121    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c02d947-b487-4eec-8a72-6300edfa5df0-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:02.185591 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.185540    2566 generic.go:356] "Generic (PLEG): container finished" podID="3c02d947-b487-4eec-8a72-6300edfa5df0" containerID="2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d" exitCode=0
Sep 16 14:41:02.185769 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.185614    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"
Sep 16 14:41:02.185769 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.185630    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j" event={"ID":"3c02d947-b487-4eec-8a72-6300edfa5df0","Type":"ContainerDied","Data":"2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d"}
Sep 16 14:41:02.185769 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.185689    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j" event={"ID":"3c02d947-b487-4eec-8a72-6300edfa5df0","Type":"ContainerDied","Data":"50feed5b955e6a386485a71926fb78d3a7045a4d7f32724a172f0b3ef8eda8d8"}
Sep 16 14:41:02.185769 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.185709    2566 scope.go:117] "RemoveContainer" containerID="2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d"
Sep 16 14:41:02.194810 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.194793    2566 scope.go:117] "RemoveContainer" containerID="2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d"
Sep 16 14:41:02.195103 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:41:02.195083    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d\": container with ID starting with 2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d not found: ID does not exist" containerID="2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d"
Sep 16 14:41:02.195153 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.195113    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d"} err="failed to get container status \"2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d\": rpc error: code = NotFound desc = could not find container \"2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d\": container with ID starting with 2d79c1c137e4fb1e927aa1ddeb460dd6b7ca0c3ea30534ddc632f5612ba8f23d not found: ID does not exist"
Sep 16 14:41:02.207050 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.207021    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"]
Sep 16 14:41:02.214198 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.214176    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-363d71/ipp-route-363d71-kserve-6b6966dbdc-2nj6j"]
Sep 16 14:41:02.984524 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:02.984493    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c02d947-b487-4eec-8a72-6300edfa5df0" path="/var/lib/kubelet/pods/3c02d947-b487-4eec-8a72-6300edfa5df0/volumes"
Sep 16 14:41:56.153384 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.153347    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"]
Sep 16 14:41:56.153877 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.153600    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5" podUID="8f42b1bb-06f8-47a8-ab77-01aafd0ed67f" containerName="payload-processing" containerID="cri-o://f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661" gracePeriod=30
Sep 16 14:41:56.164084 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.164054    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"]
Sep 16 14:41:56.164309 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.164284    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs" podUID="5742cd69-36cc-4801-adee-fc1527246299" containerName="payload-pre-processing" containerID="cri-o://6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6" gracePeriod=30
Sep 16 14:41:56.409795 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.409738    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:41:56.437186 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.437157    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:41:56.455356 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.455323    2566 generic.go:356] "Generic (PLEG): container finished" podID="8f42b1bb-06f8-47a8-ab77-01aafd0ed67f" containerID="f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661" exitCode=0
Sep 16 14:41:56.455487 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.455424    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"
Sep 16 14:41:56.455487 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.455458    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5" event={"ID":"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f","Type":"ContainerDied","Data":"f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661"}
Sep 16 14:41:56.455651 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.455513    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5" event={"ID":"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f","Type":"ContainerDied","Data":"02c1be37dd975ad820fc55e1be8e4a37731cf0aeed9eb3a2db413d48be0d5065"}
Sep 16 14:41:56.455651 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.455554    2566 scope.go:117] "RemoveContainer" containerID="f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661"
Sep 16 14:41:56.456824 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.456803    2566 generic.go:356] "Generic (PLEG): container finished" podID="5742cd69-36cc-4801-adee-fc1527246299" containerID="6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6" exitCode=0
Sep 16 14:41:56.456906 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.456836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs" event={"ID":"5742cd69-36cc-4801-adee-fc1527246299","Type":"ContainerDied","Data":"6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6"}
Sep 16 14:41:56.456906 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.456891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs" event={"ID":"5742cd69-36cc-4801-adee-fc1527246299","Type":"ContainerDied","Data":"2c7584bb350038c9324fffdf811d62b6c631146f62efd676d6a88df9b3fc8b35"}
Sep 16 14:41:56.457192 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.456843    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"
Sep 16 14:41:56.467987 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.467967    2566 scope.go:117] "RemoveContainer" containerID="f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661"
Sep 16 14:41:56.468277 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:41:56.468257    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661\": container with ID starting with f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661 not found: ID does not exist" containerID="f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661"
Sep 16 14:41:56.468374 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.468285    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661"} err="failed to get container status \"f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661\": rpc error: code = NotFound desc = could not find container \"f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661\": container with ID starting with f48c586c472486887aee9a91304efff2c4df52bda722481ae243b458e9877661 not found: ID does not exist"
Sep 16 14:41:56.468374 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.468301    2566 scope.go:117] "RemoveContainer" containerID="6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6"
Sep 16 14:41:56.477886 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.477866    2566 scope.go:117] "RemoveContainer" containerID="6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6"
Sep 16 14:41:56.478136 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:41:56.478121    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6\": container with ID starting with 6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6 not found: ID does not exist" containerID="6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6"
Sep 16 14:41:56.478189 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.478144    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6"} err="failed to get container status \"6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6\": rpc error: code = NotFound desc = could not find container \"6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6\": container with ID starting with 6fc1ce9732ef5317382bde3ff3ac6e2051fe0bc2e004e27a6aa979ff9da18fd6 not found: ID does not exist"
Sep 16 14:41:56.530473 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.530450    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-plugins-config-volume\") pod \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\" (UID: \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\") "
Sep 16 14:41:56.530620 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.530518    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddkb8\" (UniqueName: \"kubernetes.io/projected/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-kube-api-access-ddkb8\") pod \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\" (UID: \"8f42b1bb-06f8-47a8-ab77-01aafd0ed67f\") "
Sep 16 14:41:56.530620 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.530552    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdlkq\" (UniqueName: \"kubernetes.io/projected/5742cd69-36cc-4801-adee-fc1527246299-kube-api-access-gdlkq\") pod \"5742cd69-36cc-4801-adee-fc1527246299\" (UID: \"5742cd69-36cc-4801-adee-fc1527246299\") "
Sep 16 14:41:56.530620 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.530590    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5742cd69-36cc-4801-adee-fc1527246299-plugins-config-volume\") pod \"5742cd69-36cc-4801-adee-fc1527246299\" (UID: \"5742cd69-36cc-4801-adee-fc1527246299\") "
Sep 16 14:41:56.530938 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.530911    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f42b1bb-06f8-47a8-ab77-01aafd0ed67f" (UID: "8f42b1bb-06f8-47a8-ab77-01aafd0ed67f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:41:56.531035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.531012    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5742cd69-36cc-4801-adee-fc1527246299-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5742cd69-36cc-4801-adee-fc1527246299" (UID: "5742cd69-36cc-4801-adee-fc1527246299"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:41:56.532759 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.532741    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5742cd69-36cc-4801-adee-fc1527246299-kube-api-access-gdlkq" (OuterVolumeSpecName: "kube-api-access-gdlkq") pod "5742cd69-36cc-4801-adee-fc1527246299" (UID: "5742cd69-36cc-4801-adee-fc1527246299"). InnerVolumeSpecName "kube-api-access-gdlkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:41:56.532854 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.532834    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-kube-api-access-ddkb8" (OuterVolumeSpecName: "kube-api-access-ddkb8") pod "8f42b1bb-06f8-47a8-ab77-01aafd0ed67f" (UID: "8f42b1bb-06f8-47a8-ab77-01aafd0ed67f"). InnerVolumeSpecName "kube-api-access-ddkb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:41:56.631744 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.631718    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddkb8\" (UniqueName: \"kubernetes.io/projected/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-kube-api-access-ddkb8\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:56.631744 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.631740    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdlkq\" (UniqueName: \"kubernetes.io/projected/5742cd69-36cc-4801-adee-fc1527246299-kube-api-access-gdlkq\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:56.631871 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.631750    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5742cd69-36cc-4801-adee-fc1527246299-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:56.631871 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.631759    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:41:56.783637 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.783607    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"]
Sep 16 14:41:56.786298 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.786275    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f0bee-9d59464df-hwdw5"]
Sep 16 14:41:56.804375 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.804344    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"]
Sep 16 14:41:56.813541 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.813515    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f0bee-6678bdbbf8-s4hbs"]
Sep 16 14:41:56.983900 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.983862    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5742cd69-36cc-4801-adee-fc1527246299" path="/var/lib/kubelet/pods/5742cd69-36cc-4801-adee-fc1527246299/volumes"
Sep 16 14:41:56.984285 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:56.984265    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f42b1bb-06f8-47a8-ab77-01aafd0ed67f" path="/var/lib/kubelet/pods/8f42b1bb-06f8-47a8-ab77-01aafd0ed67f/volumes"
Sep 16 14:41:59.628908 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:41:59.628872    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:42:14.434234 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.434201    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"]
Sep 16 14:42:14.434828 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.434457    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph" podUID="ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed" containerName="payload-processing" containerID="cri-o://ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb" gracePeriod=30
Sep 16 14:42:14.468165 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.468136    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"]
Sep 16 14:42:14.468406 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.468379    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4" podUID="1c3e1058-31f0-47e6-ba05-191fc0934793" containerName="payload-pre-processing" containerID="cri-o://8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da" gracePeriod=30
Sep 16 14:42:14.742464 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.742438    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:42:14.745313 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.745295    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:42:14.784746 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.784723    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3e1058-31f0-47e6-ba05-191fc0934793-plugins-config-volume\") pod \"1c3e1058-31f0-47e6-ba05-191fc0934793\" (UID: \"1c3e1058-31f0-47e6-ba05-191fc0934793\") "
Sep 16 14:42:14.784867 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.784757    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbzgb\" (UniqueName: \"kubernetes.io/projected/1c3e1058-31f0-47e6-ba05-191fc0934793-kube-api-access-bbzgb\") pod \"1c3e1058-31f0-47e6-ba05-191fc0934793\" (UID: \"1c3e1058-31f0-47e6-ba05-191fc0934793\") "
Sep 16 14:42:14.784867 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.784805    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvg7h\" (UniqueName: \"kubernetes.io/projected/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-kube-api-access-hvg7h\") pod \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\" (UID: \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\") "
Sep 16 14:42:14.784950 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.784912    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-plugins-config-volume\") pod \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\" (UID: \"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed\") "
Sep 16 14:42:14.785277 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.785243    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed" (UID: "ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:42:14.785277 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.785242    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c3e1058-31f0-47e6-ba05-191fc0934793-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c3e1058-31f0-47e6-ba05-191fc0934793" (UID: "1c3e1058-31f0-47e6-ba05-191fc0934793"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:42:14.786999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.786971    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-kube-api-access-hvg7h" (OuterVolumeSpecName: "kube-api-access-hvg7h") pod "ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed" (UID: "ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed"). InnerVolumeSpecName "kube-api-access-hvg7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:42:14.787093 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.787015    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c3e1058-31f0-47e6-ba05-191fc0934793-kube-api-access-bbzgb" (OuterVolumeSpecName: "kube-api-access-bbzgb") pod "1c3e1058-31f0-47e6-ba05-191fc0934793" (UID: "1c3e1058-31f0-47e6-ba05-191fc0934793"). InnerVolumeSpecName "kube-api-access-bbzgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:42:14.886271 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.886211    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c3e1058-31f0-47e6-ba05-191fc0934793-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:42:14.886271 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.886262    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbzgb\" (UniqueName: \"kubernetes.io/projected/1c3e1058-31f0-47e6-ba05-191fc0934793-kube-api-access-bbzgb\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:42:14.886271 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.886276    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvg7h\" (UniqueName: \"kubernetes.io/projected/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-kube-api-access-hvg7h\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:42:14.886520 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:14.886288    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:42:15.549381 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.549340    2566 generic.go:356] "Generic (PLEG): container finished" podID="ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed" containerID="ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb" exitCode=0
Sep 16 14:42:15.549848 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.549386    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"
Sep 16 14:42:15.549848 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.549421    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph" event={"ID":"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed","Type":"ContainerDied","Data":"ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb"}
Sep 16 14:42:15.549848 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.549468    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph" event={"ID":"ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed","Type":"ContainerDied","Data":"a23e77cc2d4323758156ce578ac0c4a995c36938b8ce87a8bc22604c31025746"}
Sep 16 14:42:15.549848 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.549490    2566 scope.go:117] "RemoveContainer" containerID="ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb"
Sep 16 14:42:15.550978 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.550951    2566 generic.go:356] "Generic (PLEG): container finished" podID="1c3e1058-31f0-47e6-ba05-191fc0934793" containerID="8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da" exitCode=0
Sep 16 14:42:15.551080 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.550993    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"
Sep 16 14:42:15.551139 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.550994    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4" event={"ID":"1c3e1058-31f0-47e6-ba05-191fc0934793","Type":"ContainerDied","Data":"8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da"}
Sep 16 14:42:15.551192 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.551144    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4" event={"ID":"1c3e1058-31f0-47e6-ba05-191fc0934793","Type":"ContainerDied","Data":"67e63a51a0bca4ff7cc473358cf047436988f51eea20402b49cdbdd7779eaa38"}
Sep 16 14:42:15.558485 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.558463    2566 scope.go:117] "RemoveContainer" containerID="ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb"
Sep 16 14:42:15.558813 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:42:15.558792    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb\": container with ID starting with ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb not found: ID does not exist" containerID="ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb"
Sep 16 14:42:15.558873 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.558820    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb"} err="failed to get container status \"ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb\": rpc error: code = NotFound desc = could not find container \"ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb\": container with ID starting with ce6f714641d0af405cfb38ad5718001271016def4e1e0a2222cc2cb3f23c86eb not found: ID does not exist"
Sep 16 14:42:15.558873 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.558836    2566 scope.go:117] "RemoveContainer" containerID="8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da"
Sep 16 14:42:15.566229 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.566212    2566 scope.go:117] "RemoveContainer" containerID="8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da"
Sep 16 14:42:15.566479 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:42:15.566461    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da\": container with ID starting with 8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da not found: ID does not exist" containerID="8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da"
Sep 16 14:42:15.566529 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.566486    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da"} err="failed to get container status \"8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da\": rpc error: code = NotFound desc = could not find container \"8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da\": container with ID starting with 8542316292b80b7d38795a94bcc53f925546a469fcab76c744752b190f7417da not found: ID does not exist"
Sep 16 14:42:15.575887 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.575863    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"]
Sep 16 14:42:15.579433 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.579412    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c0974e-75d957dbd6-5fvz4"]
Sep 16 14:42:15.593032 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.593007    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"]
Sep 16 14:42:15.597898 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:15.597874    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c0974e-855d59b9c7-zp7ph"]
Sep 16 14:42:16.298757 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:16.298726    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:42:16.984417 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:16.984384    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c3e1058-31f0-47e6-ba05-191fc0934793" path="/var/lib/kubelet/pods/1c3e1058-31f0-47e6-ba05-191fc0934793/volumes"
Sep 16 14:42:16.984787 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:16.984766    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed" path="/var/lib/kubelet/pods/ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed/volumes"
Sep 16 14:42:33.539050 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.538965    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"]
Sep 16 14:42:33.539522 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.539263    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h" podUID="e3d31b84-6f92-4825-b8c2-9fc7c314787c" containerName="payload-pre-processing" containerID="cri-o://989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9" gracePeriod=30
Sep 16 14:42:33.549314 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.549285    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"]
Sep 16 14:42:33.549779 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.549719    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf" podUID="76f40037-a3b8-49e2-b466-9f0c8a5ba99f" containerName="payload-processing" containerID="cri-o://897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00" gracePeriod=30
Sep 16 14:42:33.791673 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.791605    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:42:33.822470 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.822446    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:42:33.845246 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.845223    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3d31b84-6f92-4825-b8c2-9fc7c314787c-plugins-config-volume\") pod \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\" (UID: \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\") "
Sep 16 14:42:33.845375 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.845287    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txp9g\" (UniqueName: \"kubernetes.io/projected/e3d31b84-6f92-4825-b8c2-9fc7c314787c-kube-api-access-txp9g\") pod \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\" (UID: \"e3d31b84-6f92-4825-b8c2-9fc7c314787c\") "
Sep 16 14:42:33.845648 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.845624    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3d31b84-6f92-4825-b8c2-9fc7c314787c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3d31b84-6f92-4825-b8c2-9fc7c314787c" (UID: "e3d31b84-6f92-4825-b8c2-9fc7c314787c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:42:33.848000 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.847962    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d31b84-6f92-4825-b8c2-9fc7c314787c-kube-api-access-txp9g" (OuterVolumeSpecName: "kube-api-access-txp9g") pod "e3d31b84-6f92-4825-b8c2-9fc7c314787c" (UID: "e3d31b84-6f92-4825-b8c2-9fc7c314787c"). InnerVolumeSpecName "kube-api-access-txp9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:42:33.946129 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.946092    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-plugins-config-volume\") pod \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\" (UID: \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\") "
Sep 16 14:42:33.946308 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.946203    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwmhm\" (UniqueName: \"kubernetes.io/projected/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-kube-api-access-cwmhm\") pod \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\" (UID: \"76f40037-a3b8-49e2-b466-9f0c8a5ba99f\") "
Sep 16 14:42:33.946352 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.946322    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3d31b84-6f92-4825-b8c2-9fc7c314787c-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:42:33.946352 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.946334    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txp9g\" (UniqueName: \"kubernetes.io/projected/e3d31b84-6f92-4825-b8c2-9fc7c314787c-kube-api-access-txp9g\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:42:33.946534 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.946509    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76f40037-a3b8-49e2-b466-9f0c8a5ba99f" (UID: "76f40037-a3b8-49e2-b466-9f0c8a5ba99f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:42:33.948408 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:33.948388    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-kube-api-access-cwmhm" (OuterVolumeSpecName: "kube-api-access-cwmhm") pod "76f40037-a3b8-49e2-b466-9f0c8a5ba99f" (UID: "76f40037-a3b8-49e2-b466-9f0c8a5ba99f"). InnerVolumeSpecName "kube-api-access-cwmhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:42:34.047861 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.047777    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwmhm\" (UniqueName: \"kubernetes.io/projected/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-kube-api-access-cwmhm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:42:34.047861 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.047808    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f40037-a3b8-49e2-b466-9f0c8a5ba99f-plugins-config-volume\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:42:34.652863 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.652824    2566 generic.go:356] "Generic (PLEG): container finished" podID="76f40037-a3b8-49e2-b466-9f0c8a5ba99f" containerID="897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00" exitCode=0
Sep 16 14:42:34.653288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.652882    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"
Sep 16 14:42:34.653288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.652912    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf" event={"ID":"76f40037-a3b8-49e2-b466-9f0c8a5ba99f","Type":"ContainerDied","Data":"897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00"}
Sep 16 14:42:34.653288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.652968    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf" event={"ID":"76f40037-a3b8-49e2-b466-9f0c8a5ba99f","Type":"ContainerDied","Data":"be25adeaf00a814f5228142c8a62c3b7d0acc972dc5b514cc42936f892b52d5f"}
Sep 16 14:42:34.653288 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.652990    2566 scope.go:117] "RemoveContainer" containerID="897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00"
Sep 16 14:42:34.654461 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.654442    2566 generic.go:356] "Generic (PLEG): container finished" podID="e3d31b84-6f92-4825-b8c2-9fc7c314787c" containerID="989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9" exitCode=0
Sep 16 14:42:34.654735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.654483    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h" event={"ID":"e3d31b84-6f92-4825-b8c2-9fc7c314787c","Type":"ContainerDied","Data":"989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9"}
Sep 16 14:42:34.654735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.654498    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"
Sep 16 14:42:34.654735 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.654521    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h" event={"ID":"e3d31b84-6f92-4825-b8c2-9fc7c314787c","Type":"ContainerDied","Data":"71a258d17d10aacf7b0245d220b41e105708c7e063e802e083faee8d2268b5a4"}
Sep 16 14:42:34.667492 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.667463    2566 scope.go:117] "RemoveContainer" containerID="897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00"
Sep 16 14:42:34.667891 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:42:34.667863    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00\": container with ID starting with 897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00 not found: ID does not exist" containerID="897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00"
Sep 16 14:42:34.667972 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.667903    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00"} err="failed to get container status \"897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00\": rpc error: code = NotFound desc = could not find container \"897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00\": container with ID starting with 897924402eb43a3bb64c1b2bdd727cb88e4aa966ce4d8b157332eb3ea46bcf00 not found: ID does not exist"
Sep 16 14:42:34.667972 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.667926    2566 scope.go:117] "RemoveContainer" containerID="989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9"
Sep 16 14:42:34.676222 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.676204    2566 scope.go:117] "RemoveContainer" containerID="989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9"
Sep 16 14:42:34.676478 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:42:34.676461    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9\": container with ID starting with 989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9 not found: ID does not exist" containerID="989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9"
Sep 16 14:42:34.676547 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.676483    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9"} err="failed to get container status \"989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9\": rpc error: code = NotFound desc = could not find container \"989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9\": container with ID starting with 989fdfa67cc5496993205295e4c46be873b711e4396844ede13a650f763917a9 not found: ID does not exist"
Sep 16 14:42:34.682091 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.682070    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"]
Sep 16 14:42:34.689984 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.689963    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d31d7c-6689c6cb6b-s6c8h"]
Sep 16 14:42:34.700865 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.700841    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"]
Sep 16 14:42:34.705119 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.705088    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d31d7c-586b75fc8c-b2trf"]
Sep 16 14:42:34.983785 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.983754    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f40037-a3b8-49e2-b466-9f0c8a5ba99f" path="/var/lib/kubelet/pods/76f40037-a3b8-49e2-b466-9f0c8a5ba99f/volumes"
Sep 16 14:42:34.984104 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:34.984092    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3d31b84-6f92-4825-b8c2-9fc7c314787c" path="/var/lib/kubelet/pods/e3d31b84-6f92-4825-b8c2-9fc7c314787c/volumes"
Sep 16 14:42:36.196158 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:36.196126    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:42:59.290621 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:42:59.290581    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:44:19.988242 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:44:19.988166    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:44:44.887832 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:44:44.887782    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:44:52.384302 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:44:52.384266    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:45:00.077407 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:45:00.077375    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:45:01.333902 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:45:01.333866    2566 scope.go:117] "RemoveContainer" containerID="309651548c238138a283083f0d7bd7ae5eceaf47a667fc4f78de259000b24a28"
Sep 16 14:45:09.689652 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:45:09.689615    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:45:21.792789 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:45:21.792754    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:45:29.690004 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:45:29.689923    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:46:01.351969 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:46:01.351921    2566 scope.go:117] "RemoveContainer" containerID="b411395cdb88dfd20d13ce5af2afef7a8d05468625e5fe52fd16ebcb7e972360"
Sep 16 14:46:01.360069 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:46:01.360050    2566 scope.go:117] "RemoveContainer" containerID="bf5c9e5e2449336ca7bd19ff19020e87297b59802c0f451d177ab762604ce4f9"
Sep 16 14:46:57.788461 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:46:57.788378    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:47:01.380589 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:01.380541    2566 scope.go:117] "RemoveContainer" containerID="2f09e2861075a63d44df8b85722bdbcef29cf65c371fdca8cc64c25ad0db18fe"
Sep 16 14:47:06.186175 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:06.186139    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:47:14.186778 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:14.186741    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:47:39.475487 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475449    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"]
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475796    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb15b4d8-1ca3-4100-bd0b-298dd4948b51" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475808    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb15b4d8-1ca3-4100-bd0b-298dd4948b51" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475821    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c3e1058-31f0-47e6-ba05-191fc0934793" containerName="payload-pre-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475827    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c3e1058-31f0-47e6-ba05-191fc0934793" containerName="payload-pre-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475836    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f42b1bb-06f8-47a8-ab77-01aafd0ed67f" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475842    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f42b1bb-06f8-47a8-ab77-01aafd0ed67f" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475858    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475863    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475870    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3d31b84-6f92-4825-b8c2-9fc7c314787c" containerName="payload-pre-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475876    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3d31b84-6f92-4825-b8c2-9fc7c314787c" containerName="payload-pre-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475881    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946bc9b8-228d-4203-835e-634c615c7a03" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475886    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="946bc9b8-228d-4203-835e-634c615c7a03" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475892    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5742cd69-36cc-4801-adee-fc1527246299" containerName="payload-pre-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475898    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5742cd69-36cc-4801-adee-fc1527246299" containerName="payload-pre-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475906    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3698ec3-8f15-4b6c-8fba-7139a502a4c7" containerName="revoke-keys"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475911    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3698ec3-8f15-4b6c-8fba-7139a502a4c7" containerName="revoke-keys"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475918    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c02d947-b487-4eec-8a72-6300edfa5df0" containerName="main"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475923    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c02d947-b487-4eec-8a72-6300edfa5df0" containerName="main"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475929    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f40037-a3b8-49e2-b466-9f0c8a5ba99f" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475934    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f40037-a3b8-49e2-b466-9f0c8a5ba99f" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475980    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffe5aa8e-00d7-4c06-9367-f6d6b70e58ed" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475988    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c3e1058-31f0-47e6-ba05-191fc0934793" containerName="payload-pre-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.475994    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5742cd69-36cc-4801-adee-fc1527246299" containerName="payload-pre-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.476001    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3d31b84-6f92-4825-b8c2-9fc7c314787c" containerName="payload-pre-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.476007    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c02d947-b487-4eec-8a72-6300edfa5df0" containerName="main"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.476011    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="946bc9b8-228d-4203-835e-634c615c7a03" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.476016    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3698ec3-8f15-4b6c-8fba-7139a502a4c7" containerName="revoke-keys"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.476020    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f40037-a3b8-49e2-b466-9f0c8a5ba99f" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.476025    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f42b1bb-06f8-47a8-ab77-01aafd0ed67f" containerName="payload-processing"
Sep 16 14:47:39.476094 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.476030    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb15b4d8-1ca3-4100-bd0b-298dd4948b51" containerName="payload-processing"
Sep 16 14:47:39.478846 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.478829    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.481309 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.481290    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-db7dede1-kserve-self-signed-certs\""
Sep 16 14:47:39.493824 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.493802    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"]
Sep 16 14:47:39.585147 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.585113    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-model-cache\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.585147 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.585160    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tls-certs\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.585366 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.585197    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tmp-dir\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.585366 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.585251    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsvsc\" (UniqueName: \"kubernetes.io/projected/e53083b3-76f0-4f5a-b251-a4899bbac3e7-kube-api-access-gsvsc\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.585366 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.585278    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-home\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.585366 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.585293    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-dshm\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.686486 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.686451    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tmp-dir\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.686686 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.686506    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsvsc\" (UniqueName: \"kubernetes.io/projected/e53083b3-76f0-4f5a-b251-a4899bbac3e7-kube-api-access-gsvsc\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.686686 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.686532    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-home\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.686686 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.686550    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-dshm\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.686686 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.686592    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-model-cache\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.686686 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.686639    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tls-certs\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.687096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.686939    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tmp-dir\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.687096 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.687032    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-model-cache\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.687241 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.687144    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-home\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.688959 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.688938    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-dshm\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.689234 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.689215    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tls-certs\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.696170 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.696144    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsvsc\" (UniqueName: \"kubernetes.io/projected/e53083b3-76f0-4f5a-b251-a4899bbac3e7-kube-api-access-gsvsc\") pod \"e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") " pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:39.789018 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:39.788937    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:40.057136 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:40.057105    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"]
Sep 16 14:47:40.060720 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:47:40.060691    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode53083b3_76f0_4f5a_b251_a4899bbac3e7.slice/crio-f6ed6d59a7fd2d40c500a98f1ceefdb2d08d919e0050605683731198efe4abc5 WatchSource:0}: Error finding container f6ed6d59a7fd2d40c500a98f1ceefdb2d08d919e0050605683731198efe4abc5: Status 404 returned error can't find the container with id f6ed6d59a7fd2d40c500a98f1ceefdb2d08d919e0050605683731198efe4abc5
Sep 16 14:47:41.019604 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:41.019548    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz" event={"ID":"e53083b3-76f0-4f5a-b251-a4899bbac3e7","Type":"ContainerStarted","Data":"599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e"}
Sep 16 14:47:41.019604 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:41.019607    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz" event={"ID":"e53083b3-76f0-4f5a-b251-a4899bbac3e7","Type":"ContainerStarted","Data":"f6ed6d59a7fd2d40c500a98f1ceefdb2d08d919e0050605683731198efe4abc5"}
Sep 16 14:47:41.039777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:41.039721    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz" podStartSLOduration=2.03970577 podStartE2EDuration="2.03970577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:47:41.038088554 +0000 UTC m=+2380.515320628" watchObservedRunningTime="2026-09-16 14:47:41.03970577 +0000 UTC m=+2380.516937844"
Sep 16 14:47:41.980689 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:41.980650    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:47:49.789541 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:49.789505    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:49.790040 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:49.789557    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:49.801943 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:49.801914    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:50.068003 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:50.067918    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:47:58.385964 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.385929    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"]
Sep 16 14:47:58.389557 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.389538    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.391991 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.391970    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-db7dede1-kserve-self-signed-certs\""
Sep 16 14:47:58.400630 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.400606    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"]
Sep 16 14:47:58.545113 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.545074    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-dshm\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.545113 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.545115    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-tmp-dir\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.545343 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.545192    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f38412e8-8cd2-4cdc-8584-004207c118c3-tls-certs\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.545343 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.545248    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc8qq\" (UniqueName: \"kubernetes.io/projected/f38412e8-8cd2-4cdc-8584-004207c118c3-kube-api-access-lc8qq\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.545343 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.545302    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-model-cache\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.545343 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.545328    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-home\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.646291 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.646214    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-model-cache\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.646291 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.646252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-home\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.646291 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.646277    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-dshm\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.646560 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.646306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-tmp-dir\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.646560 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.646346    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f38412e8-8cd2-4cdc-8584-004207c118c3-tls-certs\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.646560 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.646375    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8qq\" (UniqueName: \"kubernetes.io/projected/f38412e8-8cd2-4cdc-8584-004207c118c3-kube-api-access-lc8qq\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.646741 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.646638    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-model-cache\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.646780 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.646744    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-home\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.646846 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.646826    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-tmp-dir\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.648645 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.648625    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-dshm\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.648923 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.648906    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f38412e8-8cd2-4cdc-8584-004207c118c3-tls-certs\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.653863 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.653843    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc8qq\" (UniqueName: \"kubernetes.io/projected/f38412e8-8cd2-4cdc-8584-004207c118c3-kube-api-access-lc8qq\") pod \"e2e-same-id-b-db7dede1-kserve-c87954f95-k942s\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") " pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.700476 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.700451    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:47:58.970816 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:58.970779    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"]
Sep 16 14:47:58.974330 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:47:58.974302    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf38412e8_8cd2_4cdc_8584_004207c118c3.slice/crio-44b1f109789b545c85809adc898e3e0ae9c8c02b246e3d3ab48bf81c58cc9203 WatchSource:0}: Error finding container 44b1f109789b545c85809adc898e3e0ae9c8c02b246e3d3ab48bf81c58cc9203: Status 404 returned error can't find the container with id 44b1f109789b545c85809adc898e3e0ae9c8c02b246e3d3ab48bf81c58cc9203
Sep 16 14:47:59.098135 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:59.098099    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s" event={"ID":"f38412e8-8cd2-4cdc-8584-004207c118c3","Type":"ContainerStarted","Data":"f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208"}
Sep 16 14:47:59.098307 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:59.098143    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s" event={"ID":"f38412e8-8cd2-4cdc-8584-004207c118c3","Type":"ContainerStarted","Data":"44b1f109789b545c85809adc898e3e0ae9c8c02b246e3d3ab48bf81c58cc9203"}
Sep 16 14:47:59.122157 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:47:59.122111    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s" podStartSLOduration=1.122095165 podStartE2EDuration="1.122095165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:47:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:47:59.120766289 +0000 UTC m=+2398.597998385" watchObservedRunningTime="2026-09-16 14:47:59.122095165 +0000 UTC m=+2398.599327287"
Sep 16 14:48:01.192700 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:48:01.192659    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:48:08.701033 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:48:08.700995    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:48:08.701033 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:48:08.701045    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:48:08.713539 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:48:08.713511    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:48:09.154943 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:48:09.154869    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:48:45.376057 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:48:45.376022    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"]
Sep 16 14:48:46.243888 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:48:46.243857    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"]
Sep 16 14:48:48.585701 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:48:48.585656    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:48:57.890810 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:48:57.890776    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:49:00.395284 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.395219    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz" podUID="e53083b3-76f0-4f5a-b251-a4899bbac3e7" containerName="main" containerID="cri-o://599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e" gracePeriod=45
Sep 16 14:49:00.641039 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.641005    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:49:00.689865 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.689791    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-model-cache\") pod \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") "
Sep 16 14:49:00.689865 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.689823    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsvsc\" (UniqueName: \"kubernetes.io/projected/e53083b3-76f0-4f5a-b251-a4899bbac3e7-kube-api-access-gsvsc\") pod \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") "
Sep 16 14:49:00.689865 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.689847    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tmp-dir\") pod \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") "
Sep 16 14:49:00.690143 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.689876    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-dshm\") pod \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") "
Sep 16 14:49:00.690143 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.690005    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tls-certs\") pod \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") "
Sep 16 14:49:00.690143 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.690071    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-home\") pod \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\" (UID: \"e53083b3-76f0-4f5a-b251-a4899bbac3e7\") "
Sep 16 14:49:00.690143 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.690100    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "e53083b3-76f0-4f5a-b251-a4899bbac3e7" (UID: "e53083b3-76f0-4f5a-b251-a4899bbac3e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:49:00.690143 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.690107    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e53083b3-76f0-4f5a-b251-a4899bbac3e7" (UID: "e53083b3-76f0-4f5a-b251-a4899bbac3e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:49:00.690415 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.690355    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:00.690415 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.690355    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-home" (OuterVolumeSpecName: "home") pod "e53083b3-76f0-4f5a-b251-a4899bbac3e7" (UID: "e53083b3-76f0-4f5a-b251-a4899bbac3e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:49:00.690415 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.690374    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:00.692206 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.692182    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-dshm" (OuterVolumeSpecName: "dshm") pod "e53083b3-76f0-4f5a-b251-a4899bbac3e7" (UID: "e53083b3-76f0-4f5a-b251-a4899bbac3e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:49:00.692309 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.692258    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e53083b3-76f0-4f5a-b251-a4899bbac3e7-kube-api-access-gsvsc" (OuterVolumeSpecName: "kube-api-access-gsvsc") pod "e53083b3-76f0-4f5a-b251-a4899bbac3e7" (UID: "e53083b3-76f0-4f5a-b251-a4899bbac3e7"). InnerVolumeSpecName "kube-api-access-gsvsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:49:00.692309 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.692259    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e53083b3-76f0-4f5a-b251-a4899bbac3e7" (UID: "e53083b3-76f0-4f5a-b251-a4899bbac3e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:49:00.791355 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.791317    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:00.791355 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.791349    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsvsc\" (UniqueName: \"kubernetes.io/projected/e53083b3-76f0-4f5a-b251-a4899bbac3e7-kube-api-access-gsvsc\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:00.791355 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.791360    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e53083b3-76f0-4f5a-b251-a4899bbac3e7-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:00.791632 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:00.791369    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e53083b3-76f0-4f5a-b251-a4899bbac3e7-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:01.263636 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.263517    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s" podUID="f38412e8-8cd2-4cdc-8584-004207c118c3" containerName="main" containerID="cri-o://f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208" gracePeriod=45
Sep 16 14:49:01.377191 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.377158    2566 generic.go:356] "Generic (PLEG): container finished" podID="e53083b3-76f0-4f5a-b251-a4899bbac3e7" containerID="599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e" exitCode=0
Sep 16 14:49:01.377380 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.377204    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"
Sep 16 14:49:01.377380 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.377240    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz" event={"ID":"e53083b3-76f0-4f5a-b251-a4899bbac3e7","Type":"ContainerDied","Data":"599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e"}
Sep 16 14:49:01.377380 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.377290    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz" event={"ID":"e53083b3-76f0-4f5a-b251-a4899bbac3e7","Type":"ContainerDied","Data":"f6ed6d59a7fd2d40c500a98f1ceefdb2d08d919e0050605683731198efe4abc5"}
Sep 16 14:49:01.377380 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.377313    2566 scope.go:117] "RemoveContainer" containerID="599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e"
Sep 16 14:49:01.386523 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.386499    2566 scope.go:117] "RemoveContainer" containerID="599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e"
Sep 16 14:49:01.386874 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:49:01.386846    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e\": container with ID starting with 599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e not found: ID does not exist" containerID="599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e"
Sep 16 14:49:01.386956 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.386887    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e"} err="failed to get container status \"599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e\": rpc error: code = NotFound desc = could not find container \"599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e\": container with ID starting with 599c01e0886c14d96c160fd89fe543a8b2cdb85d1953b513a8252784b1e60b1e not found: ID does not exist"
Sep 16 14:49:01.391836 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.391810    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"]
Sep 16 14:49:01.394158 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.394137    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-db7dede1-kserve-5f967d8fcf-zmzsz"]
Sep 16 14:49:01.517879 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.517819    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:49:01.597551 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.597522    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-dshm\") pod \"f38412e8-8cd2-4cdc-8584-004207c118c3\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") "
Sep 16 14:49:01.597777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.597596    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-home\") pod \"f38412e8-8cd2-4cdc-8584-004207c118c3\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") "
Sep 16 14:49:01.597777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.597624    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-tmp-dir\") pod \"f38412e8-8cd2-4cdc-8584-004207c118c3\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") "
Sep 16 14:49:01.597777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.597643    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f38412e8-8cd2-4cdc-8584-004207c118c3-tls-certs\") pod \"f38412e8-8cd2-4cdc-8584-004207c118c3\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") "
Sep 16 14:49:01.597777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.597727    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-model-cache\") pod \"f38412e8-8cd2-4cdc-8584-004207c118c3\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") "
Sep 16 14:49:01.597777 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.597770    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lc8qq\" (UniqueName: \"kubernetes.io/projected/f38412e8-8cd2-4cdc-8584-004207c118c3-kube-api-access-lc8qq\") pod \"f38412e8-8cd2-4cdc-8584-004207c118c3\" (UID: \"f38412e8-8cd2-4cdc-8584-004207c118c3\") "
Sep 16 14:49:01.598040 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.597884    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-home" (OuterVolumeSpecName: "home") pod "f38412e8-8cd2-4cdc-8584-004207c118c3" (UID: "f38412e8-8cd2-4cdc-8584-004207c118c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:49:01.598040 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.597973    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f38412e8-8cd2-4cdc-8584-004207c118c3" (UID: "f38412e8-8cd2-4cdc-8584-004207c118c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:49:01.598040 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.597989    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "f38412e8-8cd2-4cdc-8584-004207c118c3" (UID: "f38412e8-8cd2-4cdc-8584-004207c118c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:49:01.598040 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.598038    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-home\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:01.600009 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.599982    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-dshm" (OuterVolumeSpecName: "dshm") pod "f38412e8-8cd2-4cdc-8584-004207c118c3" (UID: "f38412e8-8cd2-4cdc-8584-004207c118c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:49:01.600117 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.600030    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f38412e8-8cd2-4cdc-8584-004207c118c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f38412e8-8cd2-4cdc-8584-004207c118c3" (UID: "f38412e8-8cd2-4cdc-8584-004207c118c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:49:01.600117 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.600048    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f38412e8-8cd2-4cdc-8584-004207c118c3-kube-api-access-lc8qq" (OuterVolumeSpecName: "kube-api-access-lc8qq") pod "f38412e8-8cd2-4cdc-8584-004207c118c3" (UID: "f38412e8-8cd2-4cdc-8584-004207c118c3"). InnerVolumeSpecName "kube-api-access-lc8qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:49:01.698609 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.698555    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-dshm\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:01.698609 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.698601    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-tmp-dir\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:01.698609 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.698613    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f38412e8-8cd2-4cdc-8584-004207c118c3-tls-certs\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:01.698858 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.698624    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f38412e8-8cd2-4cdc-8584-004207c118c3-model-cache\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:01.698858 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:01.698637    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc8qq\" (UniqueName: \"kubernetes.io/projected/f38412e8-8cd2-4cdc-8584-004207c118c3-kube-api-access-lc8qq\") on node \"ip-10-0-137-213.ec2.internal\" DevicePath \"\""
Sep 16 14:49:02.382470 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.382432    2566 generic.go:356] "Generic (PLEG): container finished" podID="f38412e8-8cd2-4cdc-8584-004207c118c3" containerID="f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208" exitCode=0
Sep 16 14:49:02.382681 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.382480    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"
Sep 16 14:49:02.382681 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.382524    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s" event={"ID":"f38412e8-8cd2-4cdc-8584-004207c118c3","Type":"ContainerDied","Data":"f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208"}
Sep 16 14:49:02.382681 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.382587    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s" event={"ID":"f38412e8-8cd2-4cdc-8584-004207c118c3","Type":"ContainerDied","Data":"44b1f109789b545c85809adc898e3e0ae9c8c02b246e3d3ab48bf81c58cc9203"}
Sep 16 14:49:02.382681 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.382616    2566 scope.go:117] "RemoveContainer" containerID="f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208"
Sep 16 14:49:02.391327 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.391308    2566 scope.go:117] "RemoveContainer" containerID="f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208"
Sep 16 14:49:02.391645 ip-10-0-137-213 kubenswrapper[2566]: E0916 14:49:02.391612    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208\": container with ID starting with f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208 not found: ID does not exist" containerID="f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208"
Sep 16 14:49:02.391747 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.391643    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208"} err="failed to get container status \"f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208\": rpc error: code = NotFound desc = could not find container \"f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208\": container with ID starting with f59413f1b9ffdbd6ef61c53e9d3f3a05c29b2307cbdece1dec9ca5f3ecb7f208 not found: ID does not exist"
Sep 16 14:49:02.403329 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.403300    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"]
Sep 16 14:49:02.406987 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.406965    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-db7dede1-kserve-c87954f95-k942s"]
Sep 16 14:49:02.984678 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.984644    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e53083b3-76f0-4f5a-b251-a4899bbac3e7" path="/var/lib/kubelet/pods/e53083b3-76f0-4f5a-b251-a4899bbac3e7/volumes"
Sep 16 14:49:02.985090 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:02.985073    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f38412e8-8cd2-4cdc-8584-004207c118c3" path="/var/lib/kubelet/pods/f38412e8-8cd2-4cdc-8584-004207c118c3/volumes"
Sep 16 14:49:03.888031 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:49:03.887997    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:52:09.385365 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:09.385330    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:52:16.085672 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:16.085638    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:52:21.795361 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:21.795326    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:52:28.990139 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:28.990107    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-t2hqt"]
Sep 16 14:52:37.050045 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:37.050008    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-97f7n_c1593320-c5df-4f1b-bb35-b90738b70602/global-pull-secret-syncer/0.log"
Sep 16 14:52:37.225089 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:37.225026    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r6qxk_9cc6e861-6a80-4af5-98fa-8e19d088e530/konnectivity-agent/0.log"
Sep 16 14:52:37.303965 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:37.303877    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-213.ec2.internal_bf01562adb7ffda058010a76980228c7/haproxy/0.log"
Sep 16 14:52:42.187942 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:42.187907    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cdfbdcc49-tqltg_ba836002-b083-4a42-868b-b9d4a61b0590/authorino/0.log"
Sep 16 14:52:42.279178 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:42.279138    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-9rbqq_90a13e81-e37b-457c-aa36-4594f664bd6a/kuadrant-console-plugin/0.log"
Sep 16 14:52:42.352673 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:42.352639    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-t2hqt_010db054-aeb0-48b5-beec-9f94658ffe41/limitador/0.log"
Sep 16 14:52:43.994974 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:43.994945    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-mr884_275a6078-f9ea-4055-8683-aa9c5f5355c2/kube-state-metrics/0.log"
Sep 16 14:52:44.009371 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:44.009349    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-mr884_275a6078-f9ea-4055-8683-aa9c5f5355c2/kube-rbac-proxy-main/0.log"
Sep 16 14:52:44.031762 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:44.031722    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-mr884_275a6078-f9ea-4055-8683-aa9c5f5355c2/kube-rbac-proxy-self/0.log"
Sep 16 14:52:44.257587 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:44.257498    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ddkzc_e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0/node-exporter/0.log"
Sep 16 14:52:44.272202 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:44.272171    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ddkzc_e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0/kube-rbac-proxy/0.log"
Sep 16 14:52:44.286802 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:44.286783    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ddkzc_e91ac0d8-4e1b-481f-aad1-8dd7609e3ac0/init-textfile/0.log"
Sep 16 14:52:44.308672 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:44.308646    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-dpdhb_1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55/kube-rbac-proxy-main/0.log"
Sep 16 14:52:44.326163 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:44.326127    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-dpdhb_1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55/kube-rbac-proxy-self/0.log"
Sep 16 14:52:44.345890 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:44.345865    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-dpdhb_1ef5cfb7-33fd-4a9c-8a5c-7a4f407c9f55/openshift-state-metrics/0.log"
Sep 16 14:52:46.339855 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:46.339798    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-xzgq8_39204b25-1484-48aa-beff-c86bc7e67b1c/networking-console-plugin/0.log"
Sep 16 14:52:47.412836 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.412796    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"]
Sep 16 14:52:47.413317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.413256    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38412e8-8cd2-4cdc-8584-004207c118c3" containerName="main"
Sep 16 14:52:47.413317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.413273    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38412e8-8cd2-4cdc-8584-004207c118c3" containerName="main"
Sep 16 14:52:47.413317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.413290    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e53083b3-76f0-4f5a-b251-a4899bbac3e7" containerName="main"
Sep 16 14:52:47.413317 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.413298    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e53083b3-76f0-4f5a-b251-a4899bbac3e7" containerName="main"
Sep 16 14:52:47.413654 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.413403    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e53083b3-76f0-4f5a-b251-a4899bbac3e7" containerName="main"
Sep 16 14:52:47.413654 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.413419    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f38412e8-8cd2-4cdc-8584-004207c118c3" containerName="main"
Sep 16 14:52:47.416451 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.416431    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.418870 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.418854    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8b222\"/\"openshift-service-ca.crt\""
Sep 16 14:52:47.418960 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.418854    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8b222\"/\"default-dockercfg-bwccg\""
Sep 16 14:52:47.419881 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.419867    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8b222\"/\"kube-root-ca.crt\""
Sep 16 14:52:47.426035 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.426015    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"]
Sep 16 14:52:47.497999 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.497963    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-proc\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.498201 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.498087    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-podres\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.498201 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.498116    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-lib-modules\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.498201 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.498145    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-sys\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.498201 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.498176    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dxn2\" (UniqueName: \"kubernetes.io/projected/66d2544b-58cc-483b-877f-afcab864e36f-kube-api-access-4dxn2\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.498684 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.498655    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-g67hh_0af48f71-72de-48c4-ba30-eaff4687fddb/download-server/0.log"
Sep 16 14:52:47.598814 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.598776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-podres\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.598814 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.598816    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-lib-modules\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.599020 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.598837    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-sys\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.599020 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.598863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dxn2\" (UniqueName: \"kubernetes.io/projected/66d2544b-58cc-483b-877f-afcab864e36f-kube-api-access-4dxn2\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.599020 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.598905    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-proc\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.599020 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.598929    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-sys\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.599020 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.598953    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-podres\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.599020 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.598967    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-lib-modules\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.599020 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.598982    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66d2544b-58cc-483b-877f-afcab864e36f-proc\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.607466 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.607437    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dxn2\" (UniqueName: \"kubernetes.io/projected/66d2544b-58cc-483b-877f-afcab864e36f-kube-api-access-4dxn2\") pod \"perf-node-gather-daemonset-k7s56\" (UID: \"66d2544b-58cc-483b-877f-afcab864e36f\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.726814 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.726783    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:47.994651 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:47.994517    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"]
Sep 16 14:52:47.997117 ip-10-0-137-213 kubenswrapper[2566]: W0916 14:52:47.997077    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod66d2544b_58cc_483b_877f_afcab864e36f.slice/crio-1ff9a170512e3424c8c68bce9a7ae1c0ce139c28679d5612a385dd4b72666bd8 WatchSource:0}: Error finding container 1ff9a170512e3424c8c68bce9a7ae1c0ce139c28679d5612a385dd4b72666bd8: Status 404 returned error can't find the container with id 1ff9a170512e3424c8c68bce9a7ae1c0ce139c28679d5612a385dd4b72666bd8
Sep 16 14:52:48.356440 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:48.356340    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56" event={"ID":"66d2544b-58cc-483b-877f-afcab864e36f","Type":"ContainerStarted","Data":"baf79e70fca63512a97b300654e00fb89751db0095cfd11657cc2dee330caccc"}
Sep 16 14:52:48.356440 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:48.356390    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56" event={"ID":"66d2544b-58cc-483b-877f-afcab864e36f","Type":"ContainerStarted","Data":"1ff9a170512e3424c8c68bce9a7ae1c0ce139c28679d5612a385dd4b72666bd8"}
Sep 16 14:52:48.356440 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:48.356425    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:48.376211 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:48.376154    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56" podStartSLOduration=1.376139414 podStartE2EDuration="1.376139414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:52:48.374521589 +0000 UTC m=+2687.851753664" watchObservedRunningTime="2026-09-16 14:52:48.376139414 +0000 UTC m=+2687.853371489"
Sep 16 14:52:49.269803 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:49.269772    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9dtll_fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0/dns/0.log"
Sep 16 14:52:49.290762 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:49.290735    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9dtll_fe96b29b-d87b-4d2f-b24f-f1e8fb1573e0/kube-rbac-proxy/0.log"
Sep 16 14:52:49.425756 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:49.425714    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ptf79_c611f22c-2026-4fb5-b56b-aead114a76a0/dns-node-resolver/0.log"
Sep 16 14:52:49.988798 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:49.988771    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-587c9d68f-ljdhz_4f7b8739-851a-4200-ae5d-0dab8004fd5b/registry/0.log"
Sep 16 14:52:50.031825 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:50.031789    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kw5db_087e2244-c0c9-4141-8f5f-b94636c32bf4/node-ca/0.log"
Sep 16 14:52:51.645431 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:51.645398    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bfccdb6c-srkf4_afa151bc-3aa6-443f-ad03-987f53865732/kube-auth-proxy/0.log"
Sep 16 14:52:52.612755 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:52.612668    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mkm4b_e821d775-7157-4923-bb4f-e5a650d51f51/serve-healthcheck-canary/0.log"
Sep 16 14:52:53.202767 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:53.202724    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cbwh6_29b484d2-5062-4a43-bd41-cf88d08a6594/kube-rbac-proxy/0.log"
Sep 16 14:52:53.218165 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:53.218136    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cbwh6_29b484d2-5062-4a43-bd41-cf88d08a6594/exporter/0.log"
Sep 16 14:52:53.237659 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:53.237633    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cbwh6_29b484d2-5062-4a43-bd41-cf88d08a6594/extractor/0.log"
Sep 16 14:52:54.370420 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:54.370391    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-k7s56"
Sep 16 14:52:55.971824 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:55.971788    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d8b99777-j6bg5_362e4670-8270-46a5-9d16-843ec6470a47/manager/0.log"
Sep 16 14:52:55.986056 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:55.986032    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d8b99777-j6bg5_362e4670-8270-46a5-9d16-843ec6470a47/copy-manifests/0.log"
Sep 16 14:52:57.660926 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:57.660843    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8544d9d757-jkwgv_0da4d074-3940-4073-b4d0-f85d1ddda6f8/manager/0.log"
Sep 16 14:52:58.540761 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:52:58.540730    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bfccdb6c-srkf4_afa151bc-3aa6-443f-ad03-987f53865732/kube-auth-proxy/0.log"
Sep 16 14:53:00.776386 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:00.776331    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bfccdb6c-srkf4_afa151bc-3aa6-443f-ad03-987f53865732/kube-auth-proxy/0.log"
Sep 16 14:53:02.587876 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:02.587846    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-nmdvh_c00271b9-8282-4d7f-9b5a-7f27d8a3d49b/migrator/0.log"
Sep 16 14:53:02.602657 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:02.602628    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-nmdvh_c00271b9-8282-4d7f-9b5a-7f27d8a3d49b/graceful-termination/0.log"
Sep 16 14:53:03.986740 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:03.986716    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7lcv6_5b1d453d-626a-4d08-ad51-474bfdce1e13/kube-multus-additional-cni-plugins/0.log"
Sep 16 14:53:04.002658 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:04.002626    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7lcv6_5b1d453d-626a-4d08-ad51-474bfdce1e13/egress-router-binary-copy/0.log"
Sep 16 14:53:04.017953 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:04.017926    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7lcv6_5b1d453d-626a-4d08-ad51-474bfdce1e13/cni-plugins/0.log"
Sep 16 14:53:04.033313 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:04.033283    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7lcv6_5b1d453d-626a-4d08-ad51-474bfdce1e13/bond-cni-plugin/0.log"
Sep 16 14:53:04.050725 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:04.050690    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7lcv6_5b1d453d-626a-4d08-ad51-474bfdce1e13/routeoverride-cni/0.log"
Sep 16 14:53:04.063470 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:04.063426    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7lcv6_5b1d453d-626a-4d08-ad51-474bfdce1e13/whereabouts-cni-bincopy/0.log"
Sep 16 14:53:04.077830 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:04.077797    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7lcv6_5b1d453d-626a-4d08-ad51-474bfdce1e13/whereabouts-cni/0.log"
Sep 16 14:53:04.481461 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:04.481412    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-z8r9s_87bc3c4d-27ec-4087-9e7f-a8c7ae6c3349/kube-multus/0.log"
Sep 16 14:53:04.541587 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:04.541538    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sjjm9_e3059f22-c414-4a85-9a55-47485f8a88b3/network-metrics-daemon/0.log"
Sep 16 14:53:04.557619 ip-10-0-137-213 kubenswrapper[2566]: I0916 14:53:04.557597    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sjjm9_e3059f22-c414-4a85-9a55-47485f8a88b3/kube-rbac-proxy/0.log"